In the coming months, we will be adding many exciting new capabilities to Burp, based on the Collaborator technology. See the "Release roadmap" section towards the end of this post for more details.
This blog post looks at:
- Some important limitations of the conventional web testing model.
- How Burp Collaborator will address these limitations.
- Lots of real-world vulnerabilities that don't fit into the usual classifications.
The conventional web testing modelAt its core, conventional web security testing is fairly simple: we send payloads to the target application, and analyze responses to identify security vulnerabilities.
This model is well established and understood, and enables us to identify many kinds of security issues. Much of the work can also be automated with a degree of reliability.
However, the conventional testing model also misses lots of bugs, for example:
- We miss "super-blind" injection vulnerabilities. People often refer to "blind SQL injection", meaning only that an application doesn't return helpful error messages when a payload breaks the interpreted query. But in some situations, a successful injection has no detectable effect on application responses: that is, no difference in response contents or timing. For example, SQL injection into an asynchronous logging function is typically super-blind in this sense.
- We miss many vulnerabilities involving stored data. Bugs like stored XSS are easy to find in principle (by monitoring later responses for earlier payloads). But other stored bugs are much harder to find. For example, in stored (or second-order) SQL injection, data placed safely into a database is later read back and concatenated into a SQL query. To find bugs like this in the conventional model, we might need to brute force every combination of requests within the application, to send a payload in request 1 and look for effects via request 2.
- We miss vulnerabilities where a successful attack can only be detected in areas of the application that are not directly visible to the tester. For example, stored XSS vulnerabilities affecting admin-only pages.
- We miss many vulnerabilities involving interaction with external systems. Bugs in this category are often given labels like "server-side request forgery" and "remote file include".
Enter Burp CollaboratorBurp Collaborator augments the conventional testing model with a new component, distinct from Burp and the target application. Burp Collaborator can:
- Capture external interactions initiated by the target that are triggered by Burp's attack payloads.
- Deliver attacks back against the target in responses to those interactions.
- Enable the reliable detection of many new vulnerabilities.
- The Burp Collaborator server runs on the public web (by default).
- It uses its own dedicated domain name, and the server is registered as the authoritative DNS server for this domain.
- It provides a DNS service that answers any lookup on its registered domain (or subdomains) with its own IP address.
- It provides an HTTP/HTTPS service, and uses a valid, CA-signed, wildcard SSL certificate for its domain name.
- In future, custom implementations of various other network services will be provided, such as SMTP and FTP.
Detecting external service interaction
Detecting out-of-band resource load
Additionally, the application's processing of out-of-band content exposes some important and non-conventional attack surface. The possible ways of targeting this are described in more detail later in this post.
Detecting out-of-band XSSIn cases where out-of-band resource load is identified, the most obvious related issue to check for is out-of-band XSS:
Out-of-band XSS doesn't fall under the usual XSS classifications:
- Not reflected: the payload isn't reflected from the current in-band request to the target application.
- Not stored: the payload isn't (necessarily) stored by the target application.
Detecting "super-blind" injection bugs
Detecting out-of-band injection bugs
Detecting stored out-of-band bugs
Detecting blind stored vulnerabilities
Reporting deferred interactions
- Conventional storage and later processing of input, e.g. stored SQL injection.
- Immediate asynchronous processing, e.g. by a mail spooler.
Other network services
- Inject into mail headers and detect this via SMTP interactions with the Collaborator.
- Trigger a connection to an SMB share and capture the NTLM handshake.
- Trigger a connection to an SSH service and capture credentials.
How prevalent are these issues?
- Getting a URL from a request parameter.
- Fetching the response from the URL.
- Processing the response contents in some way.
- Storage of conventional user-supplied data.
- Later reuse in a potentially unsafe manner.
- Super-blind processing of results.
- Full Collaborator-style functionality is expensive to create, requiring a custom domain name, wildcard SSL certificate, and custom implementations of various network protocols.
- Even with a suitable Collaborator-like server deployed, manually correlating input entry points and results of stored and deferred interactions is time consuming and error-prone.
- No automated scanner currently does everything that we've described.
Manual testing tools
- Burp Collaborator client
- Burp Intruder integration
Burp Collaborator client
- Monitoring function - This will generate a unique Collaborator identifier for you to use in your own test payloads. It will poll the Collaborator and give full details of any interactions that result from using this identifier.
- Response-control function - This will let you configure the response that the Collaborator should return to targets that interact with it.
Burp Intruder integration
Security of Collaborator data
What data does the Collaborator server store?
How is retrieval of Collaborator data controlled?
- Each instance of Burp generates a securely random secret.
- Each Collaborator-related payload that Burp sends to the target application includes a random identifier that is derived from a one-way hash (cryptographic checksum) of the secret.
- Any resulting interactions with the Collaborator will include this identifier in the transmitted data (for example, in the subdomain of a DNS lookup, or the Host header of an HTTP request).
- The secret is only ever sent by Burp to the Collaborator server, to poll for details of the resulting interactions. This is done using HTTPS, unless overridden in Burp's options.
- When the Collaborator server receives a polling request, it performs the one-way hash of the submitted secret, and retrieves the details of any recorded interactions containing identifiers that are derived from that hash.
- Hence, only the instance of Burp that generated the secret is able to retrieve the details of any interactions that are triggered by its payloads.
- Details of interactions are stored in ephemeral process memory only.
- No data of any kind is in recorded in any persistent form: for example, a database or log file.
- Details of interactions are typically retrieved by Burp shortly after they occur, and are then discarded by the server.
- Details of old interactions that have not been retrieved by Burp are discarded after a fixed interval.
- There is no administrative function for viewing interaction details, only the retrieval mechanism already described.
- The Collaborator server does not by design receive any data that could be used to identify any individual Burp user (such as an account name or license key).
Private Collaborator servers
- Core Burp Collaborator platform and server.
- Support for DNS, HTTP and HTTPS services.
- Reporting of external service interaction and out-of-band resource load.
- Support for private Collaborator servers.
- Out-of-band versions of all input-based scan checks.
- Detection of various "super-blind" vulnerabilities.
- Checks for stored versions of all relevant vulnerabilities.
- Handling of deferred interactions and retrospective reporting of resulting issues.
- Support for other network service protocols, and associated test payloads.
- Manual testing tools.