The amount of digital data in the world has grown exponentially in the last few years due to factors such as widespread access to the internet, the smartphone boom, and the huge popularity of social networking sites. Collectively we produce a mind boggling 2.5 quintillion bytes of data on a daily basis and the growth rate is so high that 90% of all information ever created was produced in the last two years alone.
With these figures in mind, it is little wonder that businesses the world over are trying to tap into the potential of data. The results are rapidly having an impact on the way we all live our lives, from how our businesses are run to how we shop and even on the quality of our healthcare system.
What is Big Data?
At its heart, Big Data is the information you need to help make better decisions. In other words, it provides businesses with the ability to interrogate and utilise the huge amounts of stored data at their disposal and uncover key customer, service and application insights to improve the effectiveness of their business. Importantly, Big Data is not simply about gathering large amounts of information, but about assigning meaning to that information as to create a more predictable world.
Without access to raw data and the means to interpret and apply data, businesses cannot make truly informed decisions on how to target and engage customers. While many sectors have been slow off the mark in buying into the potential and adopting the technology to make this business promise a reality, Big Data is already paying tangible dividends in the forward thinking worlds of advertising, healthcare, mobile and finance.
How can we use it?
One of the most over-looked commodities when it comes to advertising and marketing is data, from which powerful results and conclusions can be drawn. For example, simply knowing the gender, age, and recent activity of customers on your website can be an invaluable guide to a customer group's passion points, as well as what they may be looking to purchase in the coming weeks or months.
The insight the NHS can gain from Big Data could benefit hundreds of thousands of patients and medical professionals. Big Data can give the NHS visability into every doctor's schedule and track an inventory of every hospital bed in a 300-mile radius. One of the most commonly touted criticisms of Big Data is the cost of investment. However, with public taxes being channeled into more efficient services and waiting times significantly reduced, the economic and healthcare benefits to the public are clear to see.
The benefit of Big Data to the financial sector is just coming starting to show. There have been over 18,000 flash events in capital markets trading in the last five years. Therefore there is strong economic incentive for traders to be able to detect and respond to events in a fraction of a second before such events cascade out of control. With Big Data making quantum jumps in processing and analytic speeds, a company may be able to use it to counter high frequency trading effects. In other words, will it be possible with a combination of Big Data analytics and the in-memory capabilities of modern hardware to detect and respond to events that are occurring in the nano-seconds?
The mobile revolution also adds an extra dimension to the picture - with a large proportion of phones already GPS-enabled, data on our location creates additional applications, from personal safety and monitoring services to geo-targeted advertising.
The practical applications Big Data can have in our daily lives are fascinating and give us an exciting glimpse into the future. However, to gain a truly 360 degree view of this important concept we also need to take a close look at the technology that driving it and the networks that we all rely on to make Big Data a reality.
How does it work?
Together with the data revolution comes significant challenges on the network side. All of the applications discussed above could not exist without the underlying technology to support them. Vast amounts of data are generated every day and it is imperative that all this data is backed up and stored securely. This naturally drives a need for more bandwidth into the network, which risks impacting performance in several ways.
Firstly, while the storage market has largely grown in line with the rise in global data quantities, operator networks that carry the increased data load have not. As the Big Data trend gathers momentum, a real risk exists that the network element becomes a troublesome bottleneck in the data ecosystem. As the quantity of data continues to increase, flexibility and scalability of the network will come to the fore as well.
Parallel to the capacity problem comes the challenge of network security. Particularly in the financial, healthcare and public administration sectors, much of the data transferred over both private and public networks consists of personal records falling under many countries' data protection acts. Sensitive information is also being collected through GPS and other applications. While IT managers have done a good job of securing data stored within their own corporate environments or data centres, protecting the data on operator networks remains a relatively weak point, and one which clearly needs to be addressed to enable the data revolution to continue unhindered.
The world changed because of the internet, and Big Data is a game changer on a similar, albeit more subtle scale. As more sensitive information is distributed across networks, a comprehensive IT security approach is essential. To keep up with the Big Data revolution and leverage its benefits, businesses should look at potential weak spots in the networks they rely on. This revolution will lead network operators to look at ways to ensuring that capacity and security requirements can be met and prevent the network from obstructing the potential of the data explosion and prevent the grid lock ahead on the Big Data motorway.