Big Data Security Analytics Part 4: Visualization is Key

Humans are visual hunters. Sight is our dominant sense and our brain structure is optimized for pattern recognition, these facts make data visualization one of the more powerful techniques we can apply to the big data security problem. There is also a factor of time scale, in traditional operational security monitoring you are dealing with a time period of 5-30 minutes; however, as the advanced attacker is aware of this limitation, you cannot stop "hunting" them down in your historical data, otherwise known as analytical time. 

 

Another power of data visualization is the ability to root cause analyze and remove. If you take millions of events that cover a period of months you are guaranteed that the vast majority of it will be legitimate activity. You can explore this data using multiple visual techniques and as you root cause analyze it as legitimate activity you can remove it from these visualization and slowly peel the onion back to the subset of data for which you cannot find an obvious root cause and this resulting data set will be of real interest for security events. 

 

slow and low.pngThere is an interesting phenomenon when you look at enterprise log data with visualization tool; computers have a hard time with random patterns but humans almost can't help but have bits of randomness in their activity. This is very visually recognizable in many cases and can be used to identify malicious activity. This is what makes simple visual exploration a powerful tool for identifying malicious activity in addition to algorithmic data analysis. The best approach is to conduct open ended data exploration and as you identify interesting items these can then be automated for consistent identification. 

 

Ultimately, the most powerful aspect of data visualization applied to big security data is that a data scientist is not required. A security subject matter expert has the best chance to find bad guys.

 

To explore the types of questions you can ask of your big data read the blog post: Important Questions for Big Security Data

 

Click here to learn more about HP HAVEn.

 

Thank you Chris Calvert for contributing this content.

 

Check out part 5 of this series: Big Data Security Analytics Part 5: The Challenges of Successful BDSA

Labels: haven
Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation