It has been almost impossible to avoid the phenomenon of Big Data over the past few months. Discussion about Big Data is sweeping the globe as consumers and organisations continue to create volumes of data that are literally becoming too large to handle. Ex-chief executive and current executive chairman of Google, Eric Schmidt, said back in 2010 that "Every two days, we create as much information as we did from the dawn of civilisation up until 2003." This is mind-blowing because in 2003, we were already highly dependent on the internet, laptops, email etc. So what has changed in the past decade that we are now producing over five exabytes every 48 hours?
As technology continues to adapt, we are becoming ever more reliant on technologies that make our lives easier and more productive, but at the same time consume more data than we ever imagined possible. Smart Phones and tablets make data highly available at all times but are also causing a massive 'pile up' of data. While there are many smart phones and tablets on the market, it's safe to say that Apple has been the dominant player up to now with a staggering 58 million iPads and over 175 million iPhones sold. Whilst these numbers are very impressive, let's take a look at the Apple App Store. There are over 500,000 apps available on iOS devices, which alone is a reasonable chunk of data, but the real story is the number of downloads the store has recorded. Apple are currently honing in on 25 billion downloads, that's correct, BILLION. That is a mammoth amount of data downloaded to personal devices and we can now begin to see why these huge amounts of data are beginning to stack up.
So not only are these apps creating a mountain of data, the apps themselves are responsible for creating huge datasets. Some of the most widely used apps responsible for increased data loads are social media apps such as Twitter and Facebook. Facebook allows users to upload an endless amount of photos and videos and with 800 million users worldwide, you can imagine that the servers at Facebook HQ are always running hot. The micro-blogging phenomenon Twitter, while only allowing 140 characters per Tweet, might not seem like a major Big Data contributor, but a Twitter record was broken during the recent Grammy's award ceremony. After popular singer Adele accepted her 6th Grammy for the evening, there were over 10,000 tweets sent every second mentioning the singer. That is a monstrous amount of data being created. It also demonstrates how one small 140 character tweet can be placed into the Big Data category. The Grammys ended up with a total of 13 million comments across all social media networks, further proving the power of social media.
So we have seen first-hand that the shift has already occurred and we didn't even realise it. Consumers are contributing to the shift to Big Data, but they are not the greatest contributors.
AARNet is the network responsible for sending all data between Australia's Univerisites and their Researchers and Educators all across the globe. The demands are high and we are seeing massive amounts of data being sent over the network between educational institutions. For example, telepresence video which requires high definition video in real-time, involves large amounts of data sent and received by both parties. The AARNet Network is able to host these data rich telepresence meetings between campuses thousands of kilometres with such high definition that medical students are able to participate in lectures via telepresence and have the same experience as students who are in the room. As well as video becoming an increasingly popular tool for business and educational purposes, video content is also becoming one of the most popular items for consumers to download from the internet. The University sector in Australia is one of the world leaders in Big Data transfers. Sustained transfers of data at five gigabits per second, that's right, the equivalent of seven DVD movies are being achieved as we speak between Australian Universities and their peers in the US. But it's not only the "big science" of astronomy, physics and medical that are leading the data race, some Universities are beginning to issue iPads for first year courses as compulsory and with an average textbook weighing in at 2 gig of data - the mind boggles when you multiply that by the number of text books needed by each student.
The creation of data is ballooning so rapidly that 90% of the data in the world today has been created in the last two years alone. This figure shows the extremity of the world's data growth. As technology continues to play a large role in all aspects of our lives, from communication to education; from medical treatment and entertainment to new ways of doing business, we are going to see data growth continue to explode. So the topic is no longer broadband, it's all about Big Data!
Chris Hancock, CEO AARNet
