Scientific American Article on Big Data

The October 2013 edition of Scientific American contains a fascinating article by Alex Pentland from MIT Human Dynamics Lab on Big Data and how it can be used to design better "social systems" like government and groups within society.




Pentland argues that today's cities and governments operate according to principals developed two centuries ago during the industrial revolution. These social systems now face 21st century problems such as exploding population growth and climate change and thus they need to be redesigned.


Today, we leave behind "digital bread crumbs" and with the application of Big Data we can use these data to redesign our social systems.


What are these "digital bread crumbs"?


When we buy something, that's a bread crumb. Where we go, that's another. What we eat, that's another. A FaceBook post is not a bread crumb - it's what we choose to say about the world, but it may not be a accurate reflection of what we actually did.


Pentland's team have found that high levels of person-to-person engagement always predict high levels of productitivity.


And "exploration", the extent to which the members of a group bring in new ideas from outside, is a good preidctor of innovation and creative output.


We can then use these findings to design human systems at work and in society in general. Big data allows to measure the levels of enagement and exploration before and after system design changes.


The article lists examples of call centers, marketing departments, financial traders, and healthcare insurance claimants that have all been helped by Pentland's researchers.


All great stuff. But we need safeguards, and Pentland goes on to lists a "New Deal on Data" - a series of tenets that must be adhered to when it comes to our data's privacy.


Author : Mike Shaw

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Showing results for 
Search instead for 
Do you mean 
About the Author
Mike has been with HP for 30 years. Half of that time was in R&D, mainly as an architect. The other 15 years has been spent in product manag...

Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation