A CLI crawler: goes through each link found on the page and creates a site map.
# limit output to unique URLs
$ echo "http://kali" | hakrawler -u
http://kali/
http://kali/services
http://kali/banners
http://kali/bannersExercise
http://kali/manual/
http://kali/automated/
http://kali/auth
http://kali/shells
http://kali/manual/products
http://kali/manual/reviews
http://kali/manual/testimonials
http://kali/manual
http://kali/manual/about
http://kali/shells/bind
http://kali/shells/reverse
http://kali/auth/login
