12/08/2016 06:28 BST | Updated 12/08/2017 06:12 BST

The Danger of Gorging on Advertising Data

Although more than a hundred years ago, the summer of 1914 has many similarities with now. In particular, it was a time of rapid technological change. The wireless telegraph, invented in 1896, had transformed communications - messages that once took days to convey could be transmitted instantaneously.

But that speed had a cost.

"There is abundant evidence", Stephen Kern, history professor at Ohio State University, notes "that one cause of World War I was a failure of diplomacy, and one cause of that failure was that diplomats could not cope with the volume and speed of electronic communication".

Diplomats, trained in the ponderous era of slowly-exchanged letters, were ill-equipped to deal with the demands new technology placed on them. After the assassination of Arch Duke Franz Ferdinand they simply didn't have the necessary time to generate solutions that would appease both parties. The result - a disastrous escalation of tensions.

This culminated in Austria-Hungary issuing a list of demands to Serbia on the 23rd of July giving them a mere 48 hours to respond. When Serbia's response to the tight deadline was deemed unsatisfactory by Austria, war was declared and the calamity of WWI began to unfold.

'We shape our tools and then they shape us'

This example shows that the pace of technological development sometimes outpaces our ability to adjust to it. And it's not just a historical concern. There is a similar disjuncture between knowledge and technology in marketing, caused by the volume and variety of data that is available to marketers.

The key problem is over-confidence. An increase in the volume of data sources has been shown by Paul Slovic, professor of psychology at the University of Oregon, to develop an unjustified certainty in the predictive power of the data. He ran an ingenious experiment with professional horse-racing handicap setters in which they were given a list of 88 variables that were useful in predicting a horse's performance. They then predicted the outcome of races using either 5, 10, 20, 30 or 40 of the variables.

The results were illuminating. Accuracy was the same regardless of the number of variables used. However, over-confidence grew as more data was harnessed. Experts over-estimated the importance of factors that had a limited value. It was only when five data points were used that accuracy and confidence were well calibrated.

An unbounded enthusiasm for data is dangerous and advertisers should avoid harnessing data merely because it exists. Instead, as much time, energy and effort should be expended in choosing which data sets to ignore as which to use.

Advertisers who resist this painful cull, and gorge on data, might end up regretting it.