3.4. Discovering hidden files, directories, searching for parameters of website

 

Busting/Brute Force – Finding Hidden Links

The process of searching for files and directories by dictionary enumeration involves creating a large list of keywords. These keywords are then inserted into the URL, and the server's response is monitored. The server may respond in various ways, and below are server responses and their potential meanings:

  • 200: The page/resource exists.
  • 301 and 302: Redirection to another page. There can be several reasons:
    • The requested resource exists, but authentication is required at the beginning. So, the server redirects to the login page.
    • The page is not found, so the server redirects to the homepage.
    • There can be other reasons depending on how the application's logic is configured.
  • 403: The server refuses to process the request. There can be several reasons:
    • The page/resource exists, but access is denied for certain users.
    • The page/resource does not exist on the server.
    • It could be blocked by a Web Application Firewall (WAF).
  • 404: The page is not found.
  • 500: The server failed to process the request due to a server error.

The success of this operation depends on the number of words in the dictionary. However, if the developer used a non-standard word, it's unlikely that you will find such a link.

 

Dictionaries

To start, you will need a dictionary. Kali Linux already comes with pre-installed dictionaries, which are available in the directory /usr/share/wordlists/:

Pre-installed wordlists in kali

The directory contains folders with special sets of dictionaries for various types of scanning. However, I recommend installing additional dictionaries. To do this, run the following command in the terminal:

sudo apt install seclists -y

After the installation, the dictionaries will be available in the directory /usr/share/seclists/:

View of installed Seclist in Kali

Here, you will find a wide variety of dictionaries for various types of testing.

 

Gobuster – Scanning WackoPicko

To begin, install the Gobuster program with the following command:

sudo apt install gobuster -y

For convenience, set up an environment variable that contains the link to the application you want to scan using the export command. For example:

export URL=http://10.0.2.4/WackoPicko

Now, you can run the Gobuster command with the following options:

dir: Scanning directories and files.

-u $URL: The link to the application you want to test.

-w /usr/share/wordlists/dirb/common.txt: The selected dictionary.

This will give you the following scan results:

Result of scanning website with Gobuster tool

Some requests received a server response with a status code of 403 Forbidden. This occurs when access to the requested resource is denied. However, it does not provide a definitive answer as to whether the resource exists on the server or not.

Some responses contain redirection, so you can instruct the program to follow the redirected links using the -r option:

Result of scanning website with Gobuster- redirection is enabled

Now the situation is slightly different. When the server received the request for /admin, it responded with a 500 Internal Server Error, and the length of the response message is 0, indicating that the server couldn't process the request for some reason. However, when you compare this output to the scanning results from ZAP, you will find that the /admin directory does exist but in the context of two files. The directory itself doesn't contain any standalone content. Additionally, with Gobuster, it was discovered that WackoPicko has hidden resources at the /error and /test addresses.

 

Gobuster – scanning Juice Shop

Run Juice Shop, then create a new variable with the following command:

export URL2=http://127.0.0.1:3000

After that, launch Gobuster with the specified options to scan Juice Shop:

gobuster dir -u $URL2 -w /usr/share/wordlists/dirb/common.txt

Scanning Juice Shop with Gobuster

Immediately, an error is displayed, suggesting that we whitelist either response code 200 OK or the response message length. The better option here is to filter by response message length, so we will use the --exclude-length option. Additionally, we will add the -t option for running multiple testing threads and a delay using the --delay option. This is done to reduce the scanning intensity since the application cannot handle heavy loads:

Scanning Juice Shop with Gobuster

Most of the discovered links are marked with a response code of 500 Internal Server Error. However, pay attention to the varying response message lengths. Let's open any of these links in a browser and examine the content of the server's response:

Discovered error page in Juice Shop

In the very first line, we see "Express 4.17.1," and below it, the names of installed files and directories on the server. A quick Google search for the phrase "express 4.17.1" indicates that it's a server-side JavaScript framework. As you may have already understood, we've found a vulnerability in the application caused by improper configuration. Error messages are inevitable and sometimes necessary, but they should never expose debug and configuration data.

 

Fuzzing -discovering parameters

Fuzzing is the process of sending a large number of both correct and incorrect requests to an application or server with the aim of triggering an error on the website or some non-standard behavior. We will delve into this further in the following sections when we explore various web attacks.

However, fuzzing can also be used to discover hidden files, directories, and parameters on a website. For instance, consider a URL like http://example.com?name=123, where "name" is a parameter that accepts specific values.

But what if the website contains other hidden parameters that can perform specific actions?

Let's try to find them. As an example, I have chosen the WordPress application. We will perform directory and file scanning (crawling) using OWASP ZAP or any similar program. The result looks like this:

Crawled website with ZAP

Many GET requests use various parameters, whose names are enclosed in parentheses. Let's take any GET request with a single parameter, for example, http://10.0.2.4/wordpress/?cat=1, and open it in a browser to understand what the link displays:

Viw of webpage with found hidden parameter

Now, instead of the "cat" parameter, we will try using different names and observe the server's response. We will use the program Wfuzz for this. To begin, let's assign the tested link to a variable named URL using the command:

export URL="http://10.0.2.4/wordpress/?FUZZ=1"

Please note that we replaced the parameter name with a special marker that the program will replace with various values from the dictionary. As the dictionary, we'll use /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt.

The final command will look like this:

wfuzz -c -z file,/usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt --hc 404,301 "$URL"

where:

-c displays the results in different colors.

-z file specifies the dictionary in a file.

--hc is a filter that excludes test results with the specified HTTP response codes.

Run the command, and you will immediately see a large list of identical values:

Fuzzing web application with Wfuzz tool

They are not of interest to us as the server responds the same way to a non-existent parameter. Let's filter them out. We will filter by the Chars column using the option -hh, and specify a length of 8057 after it:

Fuzzing web application with Wfuzz tool

If we compare the search results with the list of parameters in OWASP ZAP, we will find that some parameters are missing in ZAP, such as "attachment" and "tb." Let's try opening the links with these parameters:

Opening web page with hidden parameter

Opening web page with hidden parameter

It appears that the parameters do indeed exist. Now, if we perform fuzzing on a specific parameter value, we might find something interesting.

Using the Wfuzz program, we can also discover hidden directories, files, and subdomains. To do this, simply insert the marker "FUZZ" in the appropriate place in the URL and specify an appropriate dictionary.

Fuzzing capabilities are also available in OWASP ZAP and Burp Suite. We will delve deeper into the topic of fuzzing in upcoming lessons.