login

Burp Suite, the leading toolkit for web application security testing

PortSwigger Web Security Blog

Friday, 21 December 2012

Sample Burp Suite extension: Intruder payloads

This example shows how you can use an extension to:
  • Generate custom Intruder payloads
  • Apply custom processing to Intruder payloads (including built-in ones)
When an extension registers itself as an Intruder payload provider, this will be available within the Intruder UI for the user to select as the payload source for an attack. When an extension registers itself as a payload processor, the user can create a payload processing rule and select the extension's processor as the rule's action.

When Burp calls out to a payload provider to generate a payload, it passes the base value of the payload position as a parameter. This allows you to create attacks in which a whole block of serialized data is marked as the payload position, and your extension places payloads into suitable locations within that data, and re-serializes the data to create a valid request. Hence, you can use Intruder's powerful attack engine to automatically manipulate input deep within complex data structures.

This example is artificially simple, and generates two payloads: one to identify basic XSS, and one to trigger the ficititious vulnerability that was used in the previous custom scanner check example. It then uses a custom payload processor to reconstruct the serialized data structure around the custom payload.

Download the Intruder payloads extension. The download includes source code and the compiled JAR file for Java. It also includes an ASP.NET page that extends the serialization example to add some fictitious bugs so that you can test the custom payloads, and see that the two vulnerabilities are triggered. After loading the extension, you'll need to select the custom payloads as your Intruder payloads type, and add a payload processing rule that invokes the extension-provided processor. Note: the sample ASP.NET page uses the JavaScript btoa() function to perform Base64-encoding on the client side. This function is not supported by Internet Explorer, but works on most other browsers.

Thursday, 20 December 2012

Sample Burp Suite extension: custom scanner checks

In the previous example, we saw how an extension could be used to provide custom insertion points for use by Burp Scanner, enabling you to run the Scanner's built-in checks against entry points within serialized data or other formats that Burp does not natively support. In this example, we'll see how an extension can be used to futher extend the Scanner's behavior, by providing custom checks for passive and active scanning.

Custom scan checks are tightly integrated within Burp's scanning engine, and are invoked at the relevant stage for each base request and insertion point that the user sends for scanning. They can perform arbitrary processing, issue their own requests (when actively scanning), and report their own custom scan issues.

For the sake of this example, we've updated the demo serialized input application to contain two fictitious vulnerabilities that our extension can check for:
  • An information leakage vulnerability where a content management system is copying sensitive data into some application responses.
  • An input vulnerability where submitting the pipe character results in a distinctive error message, indicating an exploitable condition.
The sample extension demonstrates the following techniques:
  • Registering a custom scanner check.
  • Performing passive and active scanning when initiated by the user.
  • Using the Burp-provided IScannerInsertionPoint to construct requests for active scanning using specified payloads, without needing to understand how the insertion point works.
  • Using a Burp helper method to search responses for relevant match strings.
  • Highlighting relevant portions of requests and responses, in line with Burp's natively-generated scan issues.
  • Synchronously reporting custom scan issues in response to the relevant checks.
  • Guiding Burp on when to consolidate duplicated issues at the same URL (e.g., when the user has scanned the same item multiple times).
If you want to run this extension, you'll need to use the updated ASP.NET page, and also install the previous custom scan insertion points example, so that the active scan payload is inserted correctly into the serialized request.

Download the custom scanner checks extension. The download includes Java source code and the compiled JAR file. It also includes an ASP.NET page that extends the serialization example to add some fictitious bugs so that you can test the custom scanner check and see that the issues are reported. Note: the sample ASP.NET page uses the JavaScript btoa() function to perform Base64-encoding on the client side. This function is not supported by Internet Explorer, but works on most other browsers.

Tuesday, 18 December 2012

Sample Burp Suite extension: custom scan insertion points

In the previous example, we saw how a simple Burp extension could be used to render and edit a custom message data format, within the Burp UI. For the purpose of demonstrating this capability, we used a trivial serialization format, in which user-supplied input is Base64-encoded within a request parameter value.

That example contained a rather obvious XSS vulnerability: the raw input contained within the serialized data is echoed unfiltered in the application's response. But although this type of bug might be obvious to a human, automated scanners will not (in general) identify any kinds of input-based vulnerabilities in cases where the raw input needs to be embedded within an unsupported serialization format. Since the scanner does not understand the format, it has no means of submitting its usual scan payloads in the way that is needed for the application to unpack and process the payloads and trigger any bugs. This means that in this situation, equipped only with the previous example of a custom editor tab extension, you would be restricted to manual testing for input-based bugs, which is a tedious and time-consuming process.

The new extensibility API lets you tackle this problem by registering your extension as a provider of custom scanner insertion points. For each actively scanned request, Burp will call out to your extension, and ask it to provide any custom insertion points that are applicable to the request. Each insertion point that you provide is responsible for the job of constructing validly-formed requests for specific scan payloads. This lets your extension work with any data format, and embed the scanner's payloads within the request in the correct way.

Here, we can see Burp reporting the XSS vulnerability, which it has found via the custom "Base64-wrapped input" insertion point:


Here is the request that Burp made, and which was generated for Burp by our custom insertion point:


Here, via our custom message editor tab, is the literal scan payload that is embedded in the request:


So, with a few lines of extension code, we have taught Burp Scanner how to work with the unsupported serialization format. All of Burp's built-in scan checks can now place their payloads correctly into the application's requests, and bugs like this can be quickly found.

Download the custom scan insertion points extension. The download includes source code for Java and Python, and the compiled JAR file for Java. It also includes an ASP.NET page that implements the serialization format on the client and server side, so that you can send serialized data from your browser, send the request for active scanning within Burp, and find the vulnerability. Note: the sample ASP.NET page uses the JavaScript btoa() function to perform Base64-encoding on the client side. This function is not supported by Internet Explorer, but works on most other browsers.

Monday, 17 December 2012

Sample Burp Suite extension: custom editor tab

This extension demonstrates how you can extend Burp's own HTTP message editor to handle the display and editing of unsupported data formats. This capability can let you handle custom serialization implemented by specific applications, or any other data format that Burp does not natively support.

In the past, some extensions have handled unsupported serialization formats by hooking into Burp's HTTP stack, and modifying incoming and outgoing messages, in order to unpack and repack the serialized data. Although this approach can work, it is quite restricted in the type of data it can handle. And it is also inelegant: it would be preferable to customize Burp to understand the custom format itself, rather than tampering with the integrity of HTTP messages.

The new extensibility API lets you add custom tabs to Burp's HTTP message editor. When a message is about to be displayed, Burp will ask the tab whether it can handle the message. If so, the custom tab will be shown in the editor, and can support rendering and editing of the message within its own UI:


The sample extension uses an artificially simple serialization format: the serialized data is simply Base64-encoded within a request parameter. This example was chosen so as to keep the code that handles the serialization as simple as possible. But the format itself isn't the point: what matters is that you can now easily extend Burp to understand any format that you may encounter in a test.

As well as the new API for adding message editor tabs, this example also makes use of Burp's new helper methods, to carry out common tasks such as parsing and updating request parameters, encoding and decoding data in different formats, and conversion of data between String and byte forms.

Download the custom editor tab extension. The download includes source code for Java and Python, and the compiled JAR file for Java. It also includes an ASP.NET page that implements the serialization format on the client and server side, so that you can send serialized data from your browser, edit this on the fly within Burp, and see the effect in the server's response. Note: the sample ASP.NET page uses the JavaScript btoa() function to perform Base64-encoding on the client side. This function is not supported by Internet Explorer, but works on most other browsers.

[Really astute testers might spot a deliberate vulnerability in the sample ASP.NET page. More on that soon.]

Friday, 14 December 2012

Sample Burp Suite extension: custom logger

This extension provides something that has often been requested: a suite-wide HTTP logger within the main Burp UI. It provides a great example of how you can add some really useful functionality to Burp with a very small amount of code or effort.


The extension uses the following techniques, which are made possible by the new extensibility API:
  • It creates a custom tab within the main Burp UI, in which to display the message log.
  • It creates two instances of Burp's own HTTP message editor, in which to display the selected request and response (as in the Proxy history).
  • It provides an implementation of IMessageEditorController, which the message editors can query to obtain additional details about the displayed messages (to support context menu actions, etc.).
  • It asks Burp to customize its own UI components, in line with Burp's UI style.
  • It adds an HTTP listener, to receive details of requests and responses made by all Burp tools.
  • It uses an extension helper method to analyze the URL in each request.
In approximately 200 lines of fairly simple code, this extension adds a useful new feature, with all of the fiddly work (handling and rendering of HTTP messages) being done by Burp itself via the API.

Download the custom logger extension. The download includes source code for Java and Python, and the compiled JAR file for Java.

[Disclaimer: My Python fu is weak. In fact, this series of Burp extensions is the only Python code I've ever written. This extension is a bit more complicated than the earlier ones. Apologies to any Python heads if my code makes you cringe.]

Sample Burp Suite extension: traffic redirector

This extension demonstrates how to redirect outgoing HTTP requests from one host to another. This task might arise, for example, if you have mapped out an application which then moves to a different staging URL. By simply redirecting traffic to the new hostname, you can continue to drive your testing from the original site map.

The extension works as follows:
  • It registers itself as an HTTP listener.
  • For outgoing request messages, it retrieves the HTTP service for the request.
  • If the HTTP service host matches the "from" host, it uses a helper method to build a new HTTP service using the "to" host, and other details unchanged.
  • It updates the HTTP request with the new HTTP service.
Note: The sample code uses "host1.example.org" and "host2.example.org" as the "from" and "to" hostnames. You should edit the code to use your own hostnames before using it.

Download the traffic redirector extension. The download includes source code for Java and Python.

Thursday, 13 December 2012

Sample Burp Suite extension: event listeners

This extension demonstrates how to register listeners for various runtime events:
  • HTTP requests and responses for all Burp tools.
  • HTTP messages intercepted by the Proxy.
  • Addition of new scan issues.
  • The extension being unloaded by the user.
The sample extension simply prints a message to its output stream when an event occurs.

Registering an extension state listener is particularly important for any extension that starts background threads or opens system resources (such as files or database connections). The extension should listen for itself bring unloaded by the user, and should terminate any background threads or close any open resources when this event occurs. This good practice enables the user to fully unload the extension via the Burp UI.

Download the event listeners extension. The download includes source code for Java and Python, and the compiled JAR file for Java.

Sample Burp Suite extension: Hello World

Our first sample extension is about as basic as things can get, while actually doing something. It demonstrates the following techniques:
  • Setting the name of the extension, which will be shown to the user in the UI.
  • Obtaining the extension-specific output and error streams.
  • Writing messages to the output and error streams.
  • Writing a message to the main Burp alerts log.
  • Generating an exception to demonstrate how this is reported to the user.
Download the Hello World extension. The download includes source code for Java and Python, and the compiled JAR file for Java.

Writing your first Burp Suite extension

The new Burp Suite extensibility makes it much easier for non-programmers to create and use Burp extensions. This post explains the basics, and we'll soon be releasing a series of examples of Burp's extensibility in action.

You can create Burp extensions using Java or Python. For your first extension, you should choose the language that is most familiar to you. If you've used other compiled languages like C# or Visual Basic, then Java is probably the best place to start. If you've used other interpreted languages like Perl or Ruby, then start with Python.

Java

If you don't have one already, download and install an IDE that supports Java, such as Netbeans or Eclipse.

Create a new empty project, with whatever name you like.

Within the project, create a package called "burp".

Download the Burp Extender interface files, and put them into the folder that was created for the burp package.

Within the burp package, create a new Java class called "BurpExtender". Copy the following into the source code file:

package burp;

public class BurpExtender implements IBurpExtender
{
    public void registerExtenderCallbacks(

        IBurpExtenderCallbacks callbacks)
    {
        // your extension code here
    }
}


This empty extension does absolutely nothing at all, but you can still compile it and load it into Burp, just to see how things work.

Build the project, and find the location of the JAR file that was created by the IDE (usually in a folder called "dist").

In Burp (v1.5.01 or later), go to the Extender tool, and the Extensions tab, and add a new extension. Select the extension type "Java", and specify the location of your JAR file.

If all is well, the empty extension will load into Burp with no error messages.

If you wish, you can download a Netbeans project containing all of the code for the empty extension.

Python

You can create Python extensions using a Python-capable IDE, or you can use any text editor, such as Notepad on Windows.

Create a file, with whatever name you like, using the ".py" file extension. Copy the following into the source code file:

from burp import IBurpExtender

class BurpExtender(IBurpExtender):

    def registerExtenderCallbacks(self, callbacks):

       
        # your extension code here
       
        return


This empty extension does absolutely nothing at all, but you can still load it into Burp, just to see how things work.

Before running a Python extension, you will need to download Jython (the standalone JAR version), and configure Burp with its location (at Extender / Options / Python environment).

Then, go to the Extensions tab, and add a new extension. Select the extension type "Python", and specify the location of your file.

If all is well, the empty extension will load into Burp with no error messages.

Note: Because of the way in which Jython dynamically generates Java classes, you may encounter memory problems if you load several different Python extensions, or if you unload and reload a Python extension multiple times. If this happens, you will see an error like:

java.lang.OutOfMemoryError: PermGen space

You can avoid this problem by configuring Java to allocate more PermGen storage, by adding a XX:MaxPermSize option to the command line when starting Burp. For example:

java -XX:MaxPermSize=1G -jar burp.jar

Monday, 10 December 2012

Draft new extensibility API

The new Burp Suite extensibility framework employs a much improved API. This allows extensions to integrate much more deeply with Burp's internals, and customize Burp in ways that were previously not possible. It is also hopefully much easier for users with limited programming experience.

This post contains a brief summary of the key highlights in the new API. There are links at the end to the full interface files and API documentation.

Note: The new API is currently in draft form and is subject to change.

Event listeners

Extensions can register listeners for key runtime events:

void registerProxyListener(IProxyListener listener);

void registerHttpListener(IHttpListener listener);

void registerScannerListener(IScannerListener listener);

void registerExtensionStateListener(
    IExtensionStateListener listener);


This was previously achieved by implementing specific methods directly within the main BurpExtender class, but the new technique is more flexible.

User interface

Extensions can add their own tabs to Burp’s main window:

void addSuiteTab(ITab tab);

Extensions can create instances of Burp’s HTTP message editor, for use in their own UI:

IMessageEditor createMessageEditor(
    IMessageEditorController controller, boolean editable);


Extensions can create their own UI components and have Burp customize them according to Burp’s current UI configuration (font size, etc.):

void customizeUiComponent(Component component);

Extensions can add their own items to the context menus that appear throughout Burp:

void registerContextMenuFactory(IContextMenuFactory factory);

With the new menu support, menu items can be generated contextually, based on:
  • Location of the menu invocation
  • Any selected HTTP messages or Scanner issues
  • Any selected text
  • Editability
In editable contexts, menu items can drive changes to the underlying message.

Extensions can add their own tabs to message editors everywhere, to allow custom in-place rendering and editing of HTTP messages:

void registerMessageEditorTabFactory(
    IMessageEditorTabFactory factory);


This lets extensions handle unusual serialization formats without needing to unpack / repack messages by hooking into the HTTP stack.

Per-extension output and error streams are available:

OutputStream getStdout();
OutputStream getStderr();


The Burp user can choose whether to direct these streams to the extension’s UI (in the Extender tab), or to file, or to the Java system console.

Scanner integration

Extensions can register to provide custom scanner insertion points, for use in active scanning:

void registerScannerInsertionPointProvider(
    IScannerInsertionPointProvider provider);


For each actively scanned request, Burp will ask the extension to provide insertion points that should be used:

public interface IScannerInsertionPointProvider
{
    List<IScannerInsertionPoint> getInsertionPoints(
        IHttpRequestResponse baseRequestResponse);
}


The custom scanner insertion point handles the generation of requests during scanning:

public interface IScannerInsertionPoint
{
    String getInsertionPointName();
    String getBaseValue();
    byte[] buildRequest(byte[] payload);
    int[] getPayloadOffsets(byte[] payload);

    byte getInsertionPointType();
}


This lets extensions parse unusual serialized data formats, and make these available for effective scanning by Burp's native scan engine.

Extensions can add their own scanner checks:

void registerScannerCheck(IScannerCheck check);

Burp will invoke each registered check at the appropriate points during active and passive scanning, and the extension can report its own scan issues:

public interface IScannerCheck
{
    List<IScanIssue> doPassiveScan(
        IHttpRequestResponse baseRequestResponse);
    List<IScanIssue> doActiveScan(
        IHttpRequestResponse baseRequestResponse,
        IScannerInsertionPoint insertionPoint);
}


In active scanning, the insertion point handles the task of building HTTP requests containing your check’s payloads.

Intruder integration

Extensions can provide custom payload generators, which the user can then select from the Intruder payloads UI:

void registerIntruderPayloadGeneratorFactory(
    IIntruderPayloadGeneratorFactory factory);


Extensions can provide custom payload processors, which the user can invoke for any payload type:

void registerIntruderPayloadProcessor(
    IIntruderPayloadProcessor processor);


These methods allow extensions to deal with arbitrarily complex custom data formats, using Intruder's attack engine to manipulate data deep within serialized objects or other structures.

Session handling integration

Extensions can register their own session handling actions:

void registerSessionHandlingAction(
    ISessionHandlingAction action);


The Burp user can create rules which invoke a custom action, either on the current base request, or following execution of a macro:

public interface ISessionHandlingAction
{
    public void performAction(
            IHttpRequestResponse currentRequest,
            IHttpRequestResponse[] macroItems);
}


Custom session handling actions can issue their own requests if needed, and modify the base request as appropriate. This lets extensions work with arbitrarily complex session handling mechanisms, for example:
  • Where the parameters used for session tokens have changing names
  • Chaining tokens from prior responses made by any tool
  • Addition of timestamps, hashes, etc.
  • Modifying tokens inside serialized data

Helper methods

Burp provides a set of helper methods for tasks that frequently arise when writing a Burp extension, including:
  • Analysis of HTTP messages for headers, parameters, etc.
  • Decoding and encoding in common schemes.
  • Conversion of data between String and byte[] forms.
  • Searching for an expression within binary data.
  • Building HTTP messages based on URL, headers, body content, parameters etc.
  • Manipulation of parameters within HTTP requests.
These helpers are available via the new IExtensionHelpers interface. They will hopefully make the task of writing an extension much easier for users with limited programming experience.

Backwards compatibility

An objective for the new extensibility framework was to preserve backwards compatibility as far as possible. This has been largely achieved, although some compromises have been necessary, in the interests of easier development of new extensions.

All legacy extensions (compiled against the old interfaces) should be binary compatible with the new framework. In other words, you should be able to dynamically load existing JAR files into Burp via the Extender tool, and they should work exactly the same as previously.

If legacy extension code is compiled against the new interfaces, then some minor changes will be necessary for some extensions. This affects some usages of the IHttpRequestResponse and IScanIssue interfaces. These interfaces previously contained get/set methods for the server host, port and protocol, and also helper methods for obtaining the request URL and the response status code. These methods have now been removed, and the following replacements should be used instead:
  • There are new get/set methods using the new IHttpService interface to access all server details as a single object.
  • The request URL and response status code details are easily accessible for any HTTP request or response, via the helper methods in the new IExtensionHelpers interface.
The reason for actually removing the legacy methods from these interfaces (rather than deprecating them) is that in various new APIs, extensions must provide their own implementations of the IHttpRequestResponse and IScanIssue interfaces. Removing the superfluous methods altogether means that extensions won't need to provide dummy implementations of them.

Now, in order to achieve binary compatibility with legacy extensions, Burp does actually implement the removed methods in any instances of those interfaces that it itself generates and passes to extensions. Hence, if you really want to compile against the new interfaces without changing your legacy code, you can add the removed methods back to the new interface files and continue using them. But that isn't recommended.

Additionally, a small number of legacy methods are still present in the new interfaces but have been deprecated because superior alternatives are now available.

Full code and documentation

You can download the source code to the new interfaces, and browse the full Javadoc here.

New Burp Suite Extensibility

We will shortly be releasing Burp Suite Pro v1.5.01, which contains a new framework for extensibility in Burp.

A Short History

Burp has supported extensions for years. Burp's extensibility began its live very simple, and has evolved gradually without any overall plan.

The existing API is fairly limited, and its main uses have been for:
  • Changing HTTP messages on the fly
  • Initiating some testing tasks (e.g. to automate crawling / scanning)
The existing framework has been used to good effect to extend Burp's capabilities, for example:
  • WCF Binary Soap plugin, by GDS
  • Deflate plugin, by GDS
  • DSer plugin for JAVA serialized objects, by Attack & Defense Labs
  • w3af plugin, by David Robert

Current Limitations

The existing extensibility framework contains some significant limitations:
  • You can only run one extension at a time.
  • You can only load an extension at launch time, by modifying Burp's classpath.
  • Only the Java language is natively supported.
  • The processing of writing, compiling and loading an extension can be challenging for non-programmers.
  • The API doesn't expose very much in many areas that would be extremely useful to be able to extend. For example: Burp's user interface, the Scanner, Intruder and session handling.
  • Extensions are effectively bolt-ons, not properly integrated with Burp's internals.
In an attempt to address these limitations, several people have developed extensibility frameworks layered over Burp's, including:
  • Buby (Ruby)
  • Burp Suite Extended (Python)
  • Hiccup (Python)
  • Resty-Burp (REST / JSON)
These third-party frameworks are well designed, and address some of Burp's shortcomings. Nevertheless, it is cumbersome to need to install an additional framework, and the limitations in the current API still remain.

The efforts that people were evidently going to, in an attempt to improve on what Burp itself offered, made me realize: it's time to fix Burp natively.

Objectives for New Extensibility

The overall objectives for the new extensibility framework are:
  • Allow multiple extensions to run simultanously.
  • Support dynamic loading and unloading of extensions at runtime.
  • Support languages other than Java. In the initial release, Python is supported via the Jython interpreter.
  • Provide a much, much richer API that allows extensions to really integrate with Burp's internals.
  • Use a more future-proof API design, to allow easier enhancements in future.
  • As far as possible, ensure backwards compatibility with legacy extensions.
A key aspiration for the new extensibility is that people should find it quick and easy to create extensions mid-testing to work around all kinds of obstacles that can arise. For example, tasks like writing a Scanner check, creating a session handling action, or adding a new HTTP message analyzer to Burp's editor, should all be achievable in less time than it takes to work around an obstacle manually. With the new support for Python scripting, and a much more helpful API, even users without much programming experience will hopefully feel tempted to have a go, and so dramatically enhance the power of their testing with Burp.

User Forum

Get help from other users, at the Burp Suite User Forum:

Visit the forum ›

Copyright 2014 PortSwigger Ltd. All rights reserved.