12 IT Trends for 2012: #6 Big data, NoSQL and shared-nothing

If the phrase “the good news is you’ve got the CEO’s attention, the bad news is that you’ve got the CEO’s attention” resonates with you, then I wouldn’t want to be a DBA in 2012 unless you’re prepared to break with the past and start with a clean sheet of paper.

 

Whether it be structured, unstructured, big, small, real time, historical - data of all kinds are top-of-mind for executives everywhere and most of the CIO’s I meet with are expected to have good answers to challenging questions and are looking to anyone with a clue about data to help them get smart. Fast.

 

With all that opportunity untapped in exponentially growing data sets, there’s no doubt in my mind that capturing, classifying, extracting and protecting the value of information will be one of the hot beds of basic and applied IT research for years to come, but that doesn’t mean CIOs can afford to wait. From an IT leadership and delivery standpoint there are a number of key information “care abouts” to keep an eye on in 2012, two related to managing big data and two related to processing human information. 

 

The first, provocatively (and some would say mischievously), referred to as NoSQL. NoSQL databases, an abbreviation of “Not only SQL”, are near synonymous with high-volume large scale cloud apps such as Twitter, Etsy and Foursquare and ironically may even use SQL as their query language.  The second, so called shared-nothing storage architectures (also referred to as sharding), help improve scalability where traditional SQL databases would fail and have become increasingly important to enterprises seeking to manage ever larger data sets.

 

While not without their challenges, the performance and cost benefits are impressive enough that traditional enterprises are rapidly adopting both approaches in order to improve the performance of transactional and reporting systems without resorting to throwing more CPU (and therefor higher software license costs) at the problem.

 

Like the browser wars, there are more vendors in this space than room to name them, but HP’s Vertica, MongoDB, Cassandra, Hbase and Hadoop are good places to investigate. As Ken North points out in his article, the key takeaway for IT leaders is to ensure that architectures, management tools and practices are updated to address the unique characteristics of these new approaches. And the industry is not standing still, recently companies such as Clustrix have released SQL compatible databases that aim to do away with the complications of sharding while maintaining the familiarity and compatibility with traditional SQL databases.

 

Importantly NoSQL isn’t limited to row/column data. It can also be used for managing what Autonomy CEO Mike Lynch refers to as “human information”, that class of data that doesn’t fit neatly into a row/column database - such a query asking for all of the documents about the history or the Penguin Book company without getting back a list of books about the flightless black and white bird. 

 

Here the Intelligent Data Operating Layer (IDOL) serves as a mechanism to ingest, classify and index rich data types such as text, audio and video assets according to their content including abstract concepts such as meaning, images even abstract concepts such as sentiment and combine them with structured data types from columnar databases such as HP’s Vertica to perform real-time queries that seemed like science fiction to me only a few years ago. 

 

For example, imagine you’re running a call center and could type in a query such as “provide me a list of the top 10% of today’s customers who’ve a social media Klout of 60 or greater Recycled paper.jpgwho also became frustrated or angry during their purchase” and not only being able to proactively contact those clients but to be able to jump directly into the section of the call where they became frustrated to understand how to improve customer experience. Perhaps I’m a dork, but I think that’s unbelievably cool!

 

One person’s asset is another’s liability and that is doubly true of information. The final care-about relates to the combination of increasing expectations of enterprise transparency coupled with legal, industry and financial regulations both of which have resulted in an explosion in the cost of manually maintaining, finding and disposing of information - making it as expensive to prove innocence as it is difficult to discover guilt. NoSQL approaches that are capable of understanding human information (such as IDOL) can help save valuable time and money for both defenders and regulators alike while providing a potential early warning system to enable executives to curtail risky behaviors before they become front page news.

 

You probably already feel like you’re drowning in data, it’s important to get to grips with the changing technology landscape to ensure you’re not drowning in an incoherent mess of information management architectures too. 

 

Is data an asset or a liability for you? Do you harbor a secret nostalgia for the simple days paper and pencil

 

Photo by akeg - http://flic.kr/p/48bzjW

 

Comments
PaulMuller | ‎01-19-2012 08:54 PM

Further development of the story,  Amazon taking NoSQL into a mainstream offering Amazon Goes Back to the Future With 'NoSQL' Database

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
Paul Muller leads the global IT management evangelist team within the Software business at HP. In this role, Muller heads the team responsib...
Featured


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.