Big Data Security Analytics Part 2: Security Analytics Results From a Combination of Tools

A combination of tools

Finding the needles and examining the haystack is not achieved by a single tool. The data to be analyzed is gathered via security and business tools (IDS, FW, email server, social media scraper, etc.). These logs and captured data can be stored in a universal log management system and pushed to a SIEM such as ArcSight for real-time correlation and identification of threats.  The data can flow in raw form and as correlated SIEM output into a data system, such as Hadoop or Vertica

 

products.pngThese systems work in different ways to enable questions to be asked of the data.  “Is the event volume uncharacteristic for this time of the year?” “Was there increased security activity leading up to our new product announcement?”  “Is there any tie between our attrition patterns and different attack life cycle event volumes?” In combination with a content analysis system such as HP IDOL, you can ask “What is the likelihood that intellectual property is leaving my company?” or “Is there a negative sentiment about my company out in social media that raises my overall threat score?” Answers to these questions can be correlated with alerts from the SIEM to elevate the severity level of low priority events.  Additionally, these tools can be utilized by parts of the company outside of the security organization and they will have other pieces that fit into the puzzle.

 

Which build approach works best?

Many organizations looking to tackle the big-data problem find themselves asking "Do I find the questions I want to answer before I begin my project or do I build my big data capability and then determine what questions I want to ask?"

If you set out on a big data integration project with a small set of questions in mind, then the scope is well-defined and the success criteria are set. Funding will be easier to come by for this type of build-out. However, you may be limiting the full capabilities of a security analytics solution.

 

If you first install all of the plumbing for big data then it will be a big cost justification up front with no success criteria identified at the beginning. However, once the solution is in place, answering questions becomes very cheap. More importantly, the costs (in terms of dollars and impact to the business) of asking what-if questions drop dramatically. Simple questions and answers come very fast allowing more questions to be asked and more lessons to be learned. Instead of assuming all dimensions and facets of the answers are known of the data before the questions, a type of Socratic Method can be applied to your data exploration. The rate and methods of data generation have changed radically in the last several years. Why would we assume this rate of change to decrease?  Being able to quickly ask, and answer, all sorts of investigative questions becomes a huge competitive advantage.

 

The approach will be different depending on the needs of the organization, but the capabilities of the data analysis architecture should not be crippled.

 

Learn more about how HP HAVEn can help you.

 

Coming soon: Big Data Security Analytics Part 3: Data Science & Putting Structure to the Problem

Labels: ArcSight
Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
About the Author


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation