For those who hit the "snooze" button on earlier big data wake-up calls, consider this your espresso shot of information: Our society creates as much data in two days--approximately five exabytes of data--as all of civilization did prior to 2003. In light of that statistic, maybe they should change Moore's Law to A Little Moore's Law out of respect.
Connected devices are generating massive amounts of data, to the extent that we can soon expect the amount of data in the world to double every year. Not surprisingly, it's business that feels the growing pains of big data most acutely. Enterprises have become data dependent to make decisions, service customer relationships, foster innovation, improve operational efficiencies and, crucially gain competitive advantage.
So what's stopping you? Panic? Too Much, Too Late?
So much data and so little time; finding that nugget of information that could put you ahead can be like looking at the proverbial haystack trying to find that elusive needle.
Storing the hay is easy, but sifting through it requires a special set of tools, as well as a sufficient understanding of what you're looking for, why, and what you're going to do when you find it.
Historically, data analysis has been a story of complexity, limited capacity, elaborate tools, cryptic results, and poor distribution. Special equipment was required, only a small number of people knew how to use it, and the demand on their time was high.
Today, things have changed. New, powerful analytical solutions not only cost less to build than traditional platforms and perform more than ten times faster, but, crucially, business users can now sift through data using familiar reporting tools, gaining easy access to powerful on-demand analytics and allowing data scientists to focus on building models instead of running reports.
So how can you ensure the benefits of big data analysis are paying off across your entire organisation?
1. Ask yourself "Why"
The key to identifying which problems to tackle is to start with "why." Why are we analysing Big Data?
First, assess your strategic goals. These could be growing market share, controlling cost and risk, or understanding customer behaviour. Then, determine if using analytics will deliver value.
There are two important questions to answer: can the company use data models to derive insight, and can it act on the results? Working through this process will help determine where your organisation can realise value from Big Data analytics.
2. Changing company culture
You may have a focused plan, great execution, the right technical platform, and the ability to operationalise the results of analysis; but without accompanying cultural change, those things will only deliver a fraction of the potential value of big data analysis.
Let's go back to the haystack one more time. They have new sifting equipment that tells the hunters where the needle is, but the hunters aren't authorised to react to the information.
The best equipment can't make up for broken culture. Employees should be able to run analytics and see actionable answers on demand: a forecast of how close the sales team is to meeting this month's numbers, a customer's credit score, or a report of which advertising keywords to buy today. Armed with information, employees must also be comfortable and confident taking action before the value of the insight diminishes.
It's crucial to create a culture that rewards decisions and encourages analytics innovation, which may require modifying incentive and bonus structures. Not allowing employees to act is the most common point of failure for analytics projects - don't make that mistake. It's rarely mentioned in discussions of big data, but it can make or break an analytics initiative.
3.Maximising Results
Many companies are succeeding at their search for value in big data. They have the systems and infrastructure to capture and analyse Big Data; they have operational processes in place; and their employees have permission to act on the results. For these companies, the payoff can be dramatic.
For example, equity traders may need to buy or sell assets during the trading day to balance their portfolios, but one day's Opera feed can contain data for 500,000 to 1 million trades. If portfolio risk can only be calculated overnight, then institutions are exposed to an unquantifiable amount of risk during each trading day.
Big data analytics are ushering in a new era of predictive insight that is changing how companies operate and engage with their customers, suppliers, and employees. To take advantage of the opportunity, companies must start with the "whys," align analytics projects with business needs, and quantify the value that can be created. To realise the value, employees must have access to powerful, innovative, and proven technology, participate in the process, understand the results, and be empowered to act.
Get all of this right, and your needle will shine bright, creating competitive advantage and financial gain.