Using HP's big data solution to fix problems with complex systems

Us humans are creating ever more complex systems. Which is great. But whenever these systems go wrong, it’s becoming harder to figure out where the problem lies. Big data can help because typically, these complex systems create a lot of data that we need to analyse quickly in order to diagnose problems.
 
Let’s look at how big data is being used to fix complex problems. 
 
HP Operations Analytics is an HP IT management product that uses both HP ArcSight and HP Vertica. HP ArcSight is used for the collection of applications and systems’ log files and HP Vertica for the analysis of all the log file, event and performance information. 

 

Because of increased use of cloud, BYOD mobile devices and in the future, machine to machine data traffic, we are faced with managing applications comprised of components over which we have little control (we can’t put monitors on cloud providers’ systems, or on smart devices we don’t own).

 

IT ops analytics diagram.png

 

We thus have to “record everything”. When there is a problem, HP Operations Analytics allows us to go back in time from the point of problem until we find the point when the issue started to build up. HP Operations Analytics uses the service hierarchy to filter the data that it looks at. For example, if an online check-in app is having problems, HP Operations Analytics will just look at data associated with dependent services. 

 
That’s IT Operations Analytics. HP big data solution is also used by telcos for their operations analytics too – operations analytics on their 3g and 4g networks...

 

KDDI Mobile (second largest mobile operator in Japan) records billions of records of information about its mobile data networks. It uses this to solve performance and “dead area” problems that its customers are having. Without the use of HP Vertica, KDDI simply wasn’t able to solve the problems fast enough for customers (it’s no good telling someone why they were having problems two hours ago – they are probably no longer in the same place, and so dont now care !!). Like with HP Operations Analytics, KDDI is recording “micro-transactions” and using these to solve problems.

 

Vodafone Ireland has started using HP big data in the same way too.
 
HP Service AnywhereHP’s service desk-as-a-service product uses both HP Vertica and HP Autonomy. Let’s look at the numerous ways in which HP Service Anywhere uses the Autonomy IDOL engine:
 
HP Service Anywhere diag.png
 
1. Fuzzy plain text searching in the end user self-service portal:  You type in plain text search and get back issues, collaborations and knowledge articles. This uses the Autonomy IDOL engine so that the search terms don’t have to exactly match what is in the knowledge store. 

2. Actionable “hot topics” : This is also powered by the Autonomy IDOL engine's analytics of unstructured data. IDOL is asked to look for clusters of topics. This cluster analysis is used in two ways within HP Service Anywhere:

- To find hot social Q&A topics. This clustering information regarding social Q&A topics is then used to create knowledge-base content. This is sometimes called “ticketless IT support” - IT support doesn’t wait for people to raise tickets. Instead, they monitor social sentiment and proactively fix problems and generate content for the end-user knowledge base. HP IT uses such an approach. It monitors the internal dialog within HP regarding the services it offers and looks for clustering, using the IDOL engine. Then, again using the IDOL engine, it does sentiment analysis on the clusters. Hot negative sentiment is then flagged up and HP IT proactively works on this.

- To find hot topics in the incident database. This is then used to proactively create problem records

3. CTI integration. Based on the phone number calling into the service desk, caller’s data and the requests they opened recently are shown. But more interestingly, as the conversation takes place, the service desk agent can type in comments and then articles, open issues, and known problems are shown to help resolve the issue on the phone. This is similar to the HP ES big data product that monitors your Microsoft Lync conversations, passes the text to the IDOL engine, and uses the meaning that comes back from IDOL to fetch related documents and corporate information. 
 
HP’s Gen8 servers : HP support takes telemetric data from customers’ Gen8 servers, if the customer allows them to do so. When a customer calls with a problem, the system looks for a pattern of events prior to the problem occurring. In big data analysis speak, the subject matter expert (the support person) is teaching the predictive system what to look for. Then, when the system sees the same combination of two or three events occurring again, at any customer site, proactive action can be taken to avoid problems.  (see diagram below [note : this diagram is not using real customer data])
 
gen8 scatter.png
 
There is a large amount of telemetric data involved, and so HP Support uses the HP Vertica product to analyse this data. 
 
In the next post, I look at how customers are using HP HAVEn to optimize complex systems. (My previous post looked at the use of big data for fraud and compliance management)

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
Mike has been with HP for 30 years. Half of that time was in R&D, mainly as an architect. The other 15 years has been spent in product manag...


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation