Analysis of Larry Suto's comparative case study


In October 2007, Larry Suto released a case study entitled “Analyzing the Effectiveness and Coverage of Web Application Security Scanners,” available for reading at  The study compared the results of three commercial web application security scanners, including WebInspect.  There has been much discussion in the industry about this study (for a good example, see the “Coverage and a recent paper by L. Suto” thread at  Part of the discussion focuses on Suto’s questionable methodology & conclusions relating to application coverage, and the vagueness of his results.

Since any solid science experiment should be repeatable, SPI Labs set out to re-create Suto’s study to reasonably verify his conclusions and methodology.  In doing so we discovered significant discrepancies between our results and the results reported by Suto.  Attached is our final report (Suto_review_FINAL.pdf), where we indicate the results we received when we tested the same applications.

| ‎11-13-2007 04:33 PM
| ‎12-04-2007 11:27 AM

IBM/Watchfire released their analysis of Larry Suto's web scanner comparative review , which was

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Showing results for 
Search instead for 
Do you mean 
About the Author

Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.