THE BLOG

What Election Surprise? How Maigret Taught Me to Avoid Groupthink and See the Reality

11/05/2015 17:39 BST | Updated 11/05/2016 10:59 BST

On Thursday afternoon before the exit polls had been published I texted The Big Issue editor that I thought the Tories would win, possibly with an increased majority. He was surprised, along with everyone else I spoke to. And along with all the BBC experts who would be spending £15million of public money on waffling flannel later that election night.

It was a cavalcade of gimmicks and tricks and supposed experts who not once thought the unthinkable. So when the exit polls proved my earlier prediction the experts and commentators were confounded and confused. Paddy Ashdown even said that he would eat his hat if the exit polls were correct.

But the BBC was not alone in missing the 'unthinkable' from their thinking. The Guardian on the morning of the election had a head to imposed head shot of Miliband and Cameron; two frightened men as unsure as a BBC or Guardian expert of the outcome. They showed at the same time that Miliband had put on 3%, drawing level with Cameron.

All obvious indications pointed towards either a hung parliament, or the distinct possibility on the morrow of Ed Miliband putting a coalition together to form a government.

It is not so much that the pundits got it wrong. Rather that they had relied too heavily on their pollsters.

You might say that is what polls are about: relying on what people say. But what people say is not always an indication of what they will do.

Put it another way: when you ask people what they are going to vote it is a mass thing you are asking them; a public thing. But when they come to vote then that is a particular, private thing.

In order for the 'mass thing' to coalesce into an individual vote in the polling booth the mass thing has to be unequivocally, intoxicatingly popular. You have to be swept away by it. You have to feel a united grievance, or a unified sense of belonging. Hence in Scotland the voters tended to do what they said they were going to do. What Scotland had little of was the voter who did not know what to vote.

How did I come to the great accuracy I shared with the Big Issue editor way before there was any professional evidence to steer me?

When I heard that 20% of the voters were not sure of their choice I came to the conclusion that they would swing towards the status quo. I tried to imagine myself as one of those voters: I imagined myself going into that august school hall or parish hall, or caravan in the Tesco's' car park, as I saw on one occasion. And I imagined that I felt I wanted to vote in different ways; worried, fearful, in the end I might just have to vote for what already exists.

The fear of the unknown has bound many disparate people together for millennia. Now in a whirligig of opposing opinion people were likely to opt for some kind of normalcy. And that means going with the incumbent government, and not taking a punt on a new one.

There was a fear of fragmentation creeping into the electorate. I was convinced that with more choice than was normally on offer - the Ukip, the SNP, even the Greens - those who were undecided became fearful. Those who might want to risk it on Miliband or Farage saw this as a venture into the unknown.

Small, unrelated pieces of evidence coalesced in my thinking to convince me that you could not trust the predictors. Their very certainty was enough to warn you off believing the predictions; of a hung parliament and a splintered government of differing parties. It was presented in such an open and shut case by those in the know that it stank.

Looking at the small amount of evidence I could summon up from memory I realised that one thing was certain - having a fifth of the voters undecided meant they would go against the predicted.

Polls in the past have been right and wrong. But when they have been right there would have been other converging factors outside of the pollsters comprehension and measuring. Likewise when they have been wrong other immeasurables have come to play. What you might call 'untowards' having a decisive effect.

I remember the 1970 General Election as one that showed how wide of the mark pollsters could be. There was enormous disappointment felt by many left leaning people who were scandalised when the polls showed a Labour win, yet Conservative Heath got in.

The same happened in 1992 when Kinnock lost to Major; another open and shut, hermitically sealed kind of thinking where the unthinkable was not even allowed to be mentioned.

You could say that the evidence that turned me overnight into an expert at predicting the outcome days before the event was very flighty and very thin; it is and was. Nothing substantial could be drawn from my predictors bag of tricks, other than a feeling that people in these inclement SNP, Ukip inspired times would turn more Conservative, with a small 'c'.

And that the feeling that the 20% undecided was a godsend to Cameron, precisely because of these politically enriched times. Enriched by greater choice.

Pollsters and experts should spend the next few months or years reading George Simenon's Inspector Maigret novels. The great detective never seems to go towards the most obvious conclusions. He sniffs around, taking wrong turnings galore. But he uses a picture-building methodology that you could not see forming by our seasoned TV commenter's and experts last Thursday night.

The blind were truly leading the blind on that occasion. Too distant from the electorate, too buried in their poll evidence, they once again proved that the greatest of all experts have of late missed all the big changes. Experts in the City of London, who missed the oncoming crash; experts who missed all the abuses of the grooming scandal; right down to the Jimmy Savile affair.

We need to re-expert our experts. Intuition should take you somewhere towards understanding. Intuition, feelings, a nagging disbelief in the received wisdom of experts should inform your thinking.

As a new kind of expert I do realise, and it is a part of the thinking, that all I say can be thrown up in the air by new events. But let us embrace the simple idea that we have to listen outside of the usual box.

Interestingly, I was booked last Tuesday, two days before the election, for BBC Radio 4's lunchtime programme World at One. Among the points I wished to raise about the election was my prediction that Cameron would win with an increased majority. Alas they cancelled the slot and I did not get a chance to set the cat amongst the BBC pigeons early enough for them to do something about it. Wouldn't it have been be good for my ego if they had grafted me onto their buzzy enterprise election night and I could have argued with those that are supposedly in the know?

A little bit of history, a belief in the power of the unsure voter to upset the political apple cart, a sense that signed, sealed and delivered results in themselves are often doomed to get it wrong and a recognition that intuition and empathy for the undecided voter might bring you nearer the truth.

I would suggest that trying to understand how difficult it is to make up your mind in such a political climate goes right over the media's head. For they cannot empathise from their eyrie.

Putting yourself in the mind of the confused voter, sympathetically, role-playing that position - this is what I spent quite a bit of time doing. And it led me to believe that the experts were inexpert in their judgments.

Then I sat up much of the night to watch their expertise unravel.