Monday, November 23, 2009

[V13P] Content discovery

Burp now includes a content discovery function, similar in concept to OWASP's DirtBuster. You can access this feature by selecting a request or URL anywhere within Burp, and using the context menu to start content discovery.

Burp uses various techniques to discover content, including name guessing, web spidering, and extrapolation from naming conventions observed in use within the application. The feature is highly configurable, as shown by the available options which are explained below:

Target - These options control which directory to begin discovery from. Only items within this path and its subdirectories will be requested during the session. You can choose to discover files or directories or both, and how deep to recurse into discovered subdirectories.

Test case generation - These options control which file and directory names Burp will use when making requests to discover content. As well as built-in lists, Burp can harvest names used elsewhere within an application, and retry them at other locations, and can construct names based on discovered items, for example by cycling values in filenames containing numbers.

File extensions - You can specify a list of file extensions with which to test each possible filename. Burp can harvest file extensions observed in use within the application, and test these with every filename. When a file has been confirmed, Burp can also try a specific list of variant extensions with that filename, for example to check for old or backup versions of the same file.

Discovery engine - You can control how many threads are used for content discovery and spidering, whether file names are handled case sensitively, and how the discovery session interacts with Burp's main site map (in the target tab of the suite).

When you have configured your discovery session, you can start it from the control tab, which also provides runtime information about the actions being performed. The work is divided into numerous discrete tasks, which are prioritised according to their likelihood of quickly discovering new content, and new tasks are generated recursively as content is confirmed:

The discovery session employs its own site map, showing all of the content which has been discovered within the defined scope. If you have configured Burp to do so, newly discovered items will also be added to Burp's main site map.


Anonymous said...

Shouldn't that read dirbuster?

Anonymous said...

Any chance of being able to use third-party test case files, kind of like DirBuster letting you define your own files to search for

PortSwigger said...


Custom wordlists were on the wish-list, but didn't quite make the initial cut. Hopefully they'll get added pretty soon.

Anonymous said...

Hey, did anything new get added related to burp plugins? :-)

Erwin said...

Looking around for some time to replace Dirbuster with a better tool, since Dirbuster is missing some needed features. It looks like this new addition to Burp will fill those gaps. Can't wait to test it! Hope the custom word lists can be added soon, because unless you come up with some really good word lists I prefer to use my own ;)

c-q-r said...

In the Web Application Hacker's Handbook Chapter 4, they describe how you can use the Intruder from burpsuite to achieve "almost" the same results as you would with DirBuster. You basically create a GET message and capture it with your proxy, send it to the intruder, mark the position or file paths in the HTTP GET request and indicate the payload list you want to use to replace that marker with. (you can supply a file).