Using Web Scanners

In this lab, we'll familiarize ourselves with various web scanners used during penetration testing.

There are many web scanners included with Kali. These tools help a penetration tester find vulnerabilities in web sites and applications.

We'll start by clicking on the Kali menu icon and then clicking on the 03 - Web Application Analysis menu option, then Web Vulnerability Scanners.

Kali has a ton of options. Let's review some of the options before diving deeper into a couple commonly used scanners in the next sections of this lab.

• skipfish is a high speed security reconnaissance tool for web applications. It creates an interactive sitemap for a website by crawling the site and using dictionary based probes.
• wpscan is a vulnerability scanner specifically designed to scan WordPress based websites. It can be used to find security issues on such WordPress installations.
• whatweb is a powerful tool which helps identify what web technologies are being used for a particular website. The tool returns information on the technologies being used including version numbers, email addresses, web modules, account IDs, SQL errors and more.
• sqlmap is a full features SQL injection and vulnerability assessment tool. If can find and exploit SQL vulnerabilities on database driven websites. Sqlmap will be covered in more detail in a later lab.

In the next sections of this lab we will look in detail at the nikto and dirb web scanners.

Nikto is an open source web scanner and a very comprehensive tool which test web servers for 1000's of dangerous programs, files, vulnerabilities, and server configuration issues. Nikto is a command line tool like most tools in Kali and while it can be launched from the menu, it is better to simply open a Terminal using the nikto command.

As the output shows, this is the short version of the Help, a longer version can be displayed with nikto -H.

We'll run the command nikto -h 10.1.16.12 to execute Nikto against the LAMP server in the environment.

Nikto will begin the scan and start to display output. Once a scan is done, Nikto will tell us how many host(s) were tested.

We'll notice that Nikto has identified a few potential issues with this web server including that anti-clickjacking and XSS protection headers are not present. When Nikto finds a particularly dangerous issues, it will denote it in the output by noting "This might be interesting…" next to the item. 

Nikto can output its report in a number of different file formats including html, xml, cvs, and text. There are options to specify the format, or Nikto will assume the format based on the file extension used in the output (-o) option.

We'll run the command nikto -h 10.1.16.12 -o Desktop/nikto-results.html to execute Nikto and output to HTML.

When the command is run, we can see an html file has been created on the desktop.

Dirb is a web content scanner. It scans and crawls web servers looking for web objects, even hidden ones. It basically does this by conducting a dictionary based attack on the web server. A user can create their own dictionaries for custom attacks.

When we run the command dirb we get a list of options for help and use information on dirb.

The simplist use case is just to run dirb against either a http:// or https:// web site. Let's check out the LAMP server in the environment.

As we can see from the output of the various dirb runs, dirb uses a common.txt word list found in:

/usr/share/dirb/wordlists

We'll run the command ls -l /usr/share/dirb/wordlists to see the contents of this folder:

As we can see, dirb comes with a number of wordlist and we can also create our own. To view the contents of these list, we can use more or less.

We'll run the command more /usr/share/dirb/wordlists/common.txt to view the common.txt wordlist.

In this lab, we learned about web scanners used in penetration testing. We explored tools like skipfish, wpscan, whatweb, and sqlmap included in Kali, which help identify vulnerabilities in web applications.

We focused on two scanners: Nikto and Dirb. Nikto is a comprehensive web scanner that tests web servers for dangerous programs, files, and vulnerabilities. We ran Nikto against a LAMP server and it highlighted some potential issues like missing anti-clickjacking and XSS protection headers. We also learned how to generate reports in different formats.

Dirb, on the other hand, is a web content scanner that searches for web objects on servers, even hidden ones. We saw that it uses a wordlist called "common.txt" and explored the available wordlists.