WebInspect and Web Application Scanner Comparisons

IBM has been making some noise about their recent showing in Shay Chen’s web application scanner comparison study. While Shay’s results show a lot of things,  they don’t show that Appscan is a better solution.  Not by a long shot. 

 

For the sake of comparison, here’s how WebInspect ranked in different categories:

 

#1 – WIVET (Web Input Vector Extractor Teaser)  

#1 – Coverage features (tied)

#1 – Input Vectors (tied)

#1 – XSS (tied)

#2 – Audit Features Comparison

#2 – RFI

#4 – SQLi (the difference between 1st and 4th was .74%, and only included detections, not false positives. Otherwise, the results would have changed.)

 

The WIVET category is arguably the most important, and one that WebInspect won. If you can’t find a page, how can you test if for vulnerabilities?

 

What’s not included in Shay’s results are some scoring issues that helped us lose some points. In certain categories Shay used the WebInspect ‘All Checks’ policy to maintain consistency across all his tests. This unfortunately resulted in a number of false positives because certain checks that are included as a fail-safe mechanism do simple pattern matching as opposed to the more intelligent checks used in other WebInspect policies.  In other words, our ‘All Checks’ policy is the kitchen sink approach. We throw everything we can at an application, and some of that stuff isn’t necessarily pretty.  Our default scanning policy is the ‘Standard’ policy specifically for that reason. To his credit, Shay is fair in that he used the same criteria for every scanner. Here was his comment on the matter: 

 

“…the All Checks policy is not tagged as experimental and the consumer does not have any obvious leads that using it might affect the accuracy, and thus, I have no workaround for this issue.”

 

We can concede that point. We’d much rather Shay maintain a level playing field than change anything because we weren’t specific enough in our description.

 

It will be interesting to see the results of Shay’s next set of tests. We are most definitely looking forward to the competition. 

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
About the Author


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation