Big Data Part 1: Are Stats Guys the New High Priests?

“Big Data” has become a popular phrase for describing the immensity of information collecting on servers around the world. Recently, YouTube announced that an hour of video is being uploaded to its servers per second. To get a picture of what this looks like, there’s  a great webpage called onehourpersecond.com which helps illustrate this. Similarly, Economist magazine, in an article titled “Building with Big Data” lists more huge examples: “Last year people stored enough data to fill 60,000 Libraries of Congress. The world’s 4 billion mobile-phone users (12% of whom own smartphones) have turned themselves into data-streams.”

The biggest companies have been using Big Data to improve business practice for several years. For example, the article mentioned above lists three examples: “Tesco, a British retailer, collects 1.5 billion nuggets of data every month and uses them to adjust prices and promotions. Williams-Sonoma, an American retailer, uses its knowledge of its 60m customers … to produce different iterations of its catalogue. Amazon, an online retailer, has claimed that 30% of its sales are generated by its recommendation engine (‘you may also like’).” Big Data changes things, and Joe Hellerstein accurately calls this “the industrial revolution of data.”

We all have a foggy notion that we live in a world where Big Data is becoming more and more important, but just how big is big? Let me give you some perspective of how much information is actually being produced, compliments of this great infographic from Mashable:

Even with this handy-dandy infographic the human mind can barely fathom the impact of Big Data. Wired magazine makes this point well: “Google and like-minded companies are … the children of the Petabyte Age. The Petabyte Age is different because more is different. Kilobytes were stored on floppy disks. Megabytes were stored on hard disks. Terabytes were stored in disk arrays. Petabytes are stored in the cloud. As we moved along that progression, we went from the folder analogy to the file cabinet analogy to the library analogy to — well, at petabytes we ran out of organizational analogies.” Data has become so big, we have to resort to using terms like “the cloud” to describe it.

When data gets this big it requires more complex technology to manage it; after all, most of us don’t have a box of floppy drives in a drawer anymore. In Technopoly, Neil Postman explains how this can lead to a deadly cycle: “Attend any conference on telecommunications or computer technology, and you will be attending a celebration of innovative machinery that generates, stores, and distributes more information, more conveniently, at greater speeds than ever before. This is the elevation of information to a metaphysical status: information as both the means and end of human creativity.”

It shouldn’t surprise us then to find statements as strong as this: “‘If I can operate Google, I can find anything,’ said Alan Cohen, then vice president of Airespace, which sells wireless technology. ‘Google is like God. God is wireless, God is everywhere, God sees everything. Any questions in the world, you ask Google.'” It also shouldn’t surprise us that there is actually a  religion, whether farcical or not, which claims that Google actually is God. The religion is called Googlism, and it states: “We at the Church of Google believe the search engine Google is the closest humankind has ever come to directly experiencing an actual God (as typically defined). We believe there is much more evidence in favour of Google’s divinity than there is for the divinity of other more traditional gods.”

If Google is our new God, then the new high priests are the stats guys. “Hal Varian, Google’s chief economist, predicts that the job of statistician will become the ‘sexiest’ around. Data, he explains, are widely available; what is scarce is the ability to extract wisdom from them.” Those who can make Big Data mean something for our everyday lives, or who can make it produce profit for business will hold the power as we find out what the information revolution means for our future.

This blog series will highlight some of the ways Big Data is being harnessed in ways that make it tangible and accessible for the average person. Stay tuned for “Big Data Part 2: The Gurus of Big Data“.

Enhanced by Zemanta
Advertisements

2 responses to “Big Data Part 1: Are Stats Guys the New High Priests?

  • Statistics for the masses | ReStreaming

    […] Big Data Part 1: Are Stats Guys the New High Priests? (joshchalmers.wordpress.com) […]

  • RobL

    If Google is God then Silicon Valley must be the Heavenly City! Alice Marwick (www.tiara.org) has researched this and says today’s tech trends are shaped by the unique worldview of Silicon Valley and libertarianism. Moreover, people who don’t follow these trends actually pay a social cost from those who can’t believe someone wouldn’t want to use Google or Apple products and services. Thus we have the church (lit. idolatry) of GoogApple!

Please comment, but note that I reserve the right to delete comments I don't feel are helpful. If in doubt, read my comments policy.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: