Yesterday, Jon Cruddas previewed the results of "an objective, empirical analysis" of why Labour lost in May. The analysis concludes that Labour lost because voters supported austerity. This interpretation was, for most part, uncritically echoed by the media, for instance in the Guardian.
In his piece for LabourList, Cruddas writes:
"The first hard truth is that the Tories didn't win despite austerity, they won because of it. Voters did not reject Labour because they saw it as austerity lite. Voters rejected Labour because they perceived the Party as anti-austerity lite. 58% agree that, 'we must live within our means so cutting the deficit is the top priority'. Just 16% disagree. Almost all Tories and a majority of Lib Dems and Ukip voters agree."
But the real "hard truth" is that neither Jon Cruddas, nor anyone else who worked on this report, knows and can know why Labour lost in May. Cruddas might have a hypothesis about why Labour lost, he might believe he knows the answer as to why Labour lost (as Jeremy Corbyn believes and claims the opposite), but the figures from the "independent report" published yesterday do little to enlighten this debate. Instead, Cruddas pretends to offer hard evidence, numbers, empirics, that, under scrutiny, turn out to be rather useless at addressing the question of whether Labour's position on austerity caused voters to reelect David Cameron.
In fact, opinion survey data of the kind used in this report does not allow for the strong causal inference that Jon Cruddas makes. It is worrying that one would take a cross-tab or a bar-chart from an opinion survey that shows some correlation between an issue-opinion and reported vote choice and argue, without blinking, that this would constitute "objective empirical" evidence in favour of a particular causal hypothesis. Honestly, I do not know, and I know no political scientist who would claim that they knew the cause of Labour's defeat. People hold different political views and interpret the election result through that very prism, and that is entirely legitimate. What upsets me is when people misuse "objective" statistics relying on the insufficient training of journalists to spot basic flaws in causal inference.
So what is so fundamentally wrong with this report? The problem with this report is not that I do not trust the reported numbers, or that I think that opinion polls are useless. I have no reason to believe that the data has not been collected using appropriate sampling methods. I also do not think that the polling disaster that happened in May has much bearing on this question at all. The problem lies in the interpretation of the data as providing evidence in favour of causation. Cruddas claims that voters voted Conservative because of pro-austerity attitudes. But the data is ill suited to even address his claim. Why? The data is what survey researchers call a cross-section consisting of interviews with a representative sample of voters at one single point in time (though the task would not even be much easier if it consisted of multiple observations of the same respondents). What Cruddas observes is a correlation between attitudes towards austerity and vote choice. Every person who has ever attended statistics 101 should know that "correlation does not equal causation". What does this mean? It means that only because there is a relationship in the data between one variable (attitudes towards austerity) and another (vote choice), this does not mean that one caused the other. In fact, there is much evidence for the opposite view, namely that vote choice or party support affects which attitudes respondents report on opinion surveys. But that is the topic for another post.
The "hard truth" is that there is a (close to infinite) number of other, alternative explanations as to why Labour lost in May that are both correlated with attitudes towards austerity and vote choice and that cannot be ruled out on the basis of this data. Take only one example: Ed Miliband's personality ratings. In fact, there are many questions one could and one could not think of that would show similar patterns. Many other variable or combination of variables could have caused Labour's defeat. To cut a long story short, trying to determine the cause of the defeat through the use of opinion surveys (or any other objective empirical method) is (and will turn out to be) a hopeless exercise that is not objective, but intrinsically political. As the famous statistician Paul W. Holland famously concluded, science can only ever hope to identify the effect of a cause, but it will not be able to identify (ex-post) the cause of an observed effect or event. Most political scientists would agree that using cross-sectional observational data to look at the cause of Labour's defeat is indeed an impossible undertaking.
One should have thought that Jon Cruddas, who is known as an intellectual within the Labour Party, and the people working in the associated think tanks that support this report, know that their case is built on sand. While I usually applaud the use of data instead of political commentary, in this case it appears that existing patterns in the data have simply been (mis-)used to make a political point.