The authors created their own test application containing a wide variety of vulnerabilities and crawling challenges, and carried out what appears to be a very detailed and rigorous analysis of each scanner's performance against this application.
Scanners were scored based on their ability to identify different types of vulnerabilities in different scanning modes. The overall scores, together with the prices of each scanner, were as follows:
- There are whole classes of vulnerabilities that cannot be detected by the state-of-the-art scanners, including weak passwords, broken access controls and logic flaws.
- The crawling of modern web applications can be a serious challenge for today’s web vulnerability scanners, due to incomplete support for common client-side technologies and the complex stateful nature of today's applications.
- There is no strong correlation between price and capability, as some of the free or very cost-effective scanners performed as well as scanners that cost thousands of dollars.
I was, of course, pleased to see this recognition of Burp Scanner's capabilities, and the above comparison of scanners' performance versus price should make interesting reading for anyone who is deciding which products to spend their money on. Rest assured, I'll be going through the raw results from this survey in detail, and looking at ways to make Burp even more effective.