Re: Parsing binary columns from WebInspect database (199 Views)
Reply
Occasional Visitor
RuneAL
Posts: 1
Registered: ‎03-15-2010
Message 1 of 4 (283 Views)

Parsing binary columns from WebInspect database

Hi



I am working on a system that needs to take data from WebInspect result database, and present them in different ways. We have looked a bit into the data, that WebInspect produces, and we are quite interested in what kind of data that hides inside some of your binary areas of the database that WebInspect uses.



In particular we are very interested in 3 columns in 2 views:

SessionDetailView, column RequestBytes and AttackDefinition
ResponseDetailView, column ResponseBytes


The problem is that we don’t know what type of data that resides in these binary columns. We tried converting them all to txt files, and we got very mixed results.

RequestBytes was pure Text/HTML request headers, and could be read and parsed easily. But ResponseBytes is quite different. Some of the responses could be parsed out to be pure HTML responses. But sometimes the text comes out quite random, like it is another type, perhaps a compressed file or a picture perhaps. And sometimes we can parse it out to be, what looks like, Chinese characters.

Is there any way to identify what kind of content there is inside ResponseBytes, so that it is possible for us to read it?

Finally we have AttackDefinition. This is mixed with XML, Text, what looks like headers and various data unreadable by normal eyes. Could we get a bit more information on what kind of data there is in that column?

-     Rune Liljegren

Respected Contributor
HansEnders
Posts: 613
Registered: ‎07-01-2008
Message 2 of 4 (283 Views)

Re: Parsing binary columns from WebInspect database

Unfortunately the WebInspect schema has never been released internally or otherwise.  As you discovered, some of the tables offer raw text but others are encoded or compressed.  Product Management and Development reserve the right to alter that schema and the table contents at any time in the continuing development of the product and expansion of its audit engines.


You may be better served using the Full Export (File menu > Export > Scan Details > Full) and mapping those XML tags to your own database fields.  I have attached a study on the Full export from WebInspect 8.  The normal UI scans do not have an auto-export option, but the CLI scans do offer this option as flag "-ea".


-- Habeas Data
Respected Contributor
HansEnders
Posts: 613
Registered: ‎07-01-2008
Message 3 of 4 (199 Views)

Re: Parsing binary columns from WebInspect database

It seems that a migration of this forum dropped the aforementioned attachment.  Here is that ZIP attachment describing the XML data found when exporting the Scan Details > Full in WebInspect 8.1.

 

Also, a review of the WebInspect 9.30 Help guide indicates that it is the {-eb} option, not the {-ea} option, that will auto-export the scan's Full details upon completion of a scripted scan.


-- Habeas Data
Respected Contributor
HansEnders
Posts: 613
Registered: ‎07-01-2008
Message 4 of 4 (197 Views)

Re: Parsing binary columns from WebInspect database

And.....    attached is the similar data study on the Full scan details export from WebInspect 9.30.93.  It is largely identical to the earlier formats, with the exception of the <Classifications> tag added with the WebInspect 9.20 release.


-- Habeas Data
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.