15/10/2014 03:10 BST | Updated 14/12/2014 05:59 GMT

From Essex to Manchester, the Ukip Surge Signals the Death Knell of Class-Based Voting

"We are the only party that can get big vote shares in Tory heartlands and in Labour heartlands. No other party crosses... those old divides of left and right and the divides of class". This was Nigel Farage's core message, as he celebrated the election of Douglas Carswell as Ukip's first ever MP and the pushing of Labour to a close second in the Greater Manchester seat of Heywood and Middleton - won by just 617 votes.

The latter by-election in particular sums up the bitter truth of Farage's words. There was a 36% swing in favour of Ukip, in a constituency which has traditionally been one of Labour's working-class strongholds (cantering home with a majority of 4,000, even in 1983). The Ukip appeal, it would seem, really does transcend social class, and there can now be no doubt that a revolt has taken place on the left. In strictly policy terms the purple peril may speak to and for the right, but it is sustained by a significant cohort of former Labour voters.

This paradox is, for many on the left, almost unbearably acute. People in deprived areas appear to be handing Farage - a public school-educated banker - the stick with which to beat them. For those who were bewildered in the 1980s by people appearing to vote "against their interests", the prospect of potential Ukip victories in areas like Heywood and Middleton will be baffling. What has it come to that working-class communities would choose, in large numbers, to vote for a party to the right of the Tories? Discussion in the media over the weekend seemed, at points, to verge on despair - a feeling of 'Where do we go from here?'

I would argue that this feeling is misplaced. I do not believe that we are seeing the 'End of Politics'. I do, however, believe that we have reached the culmination of a steady, 40-year shift away from class-based voting. The two will feel, to many, like they are very much the same thing. Indeed, if the main parties do not face up to the change that has taken place, they could become so.

The proportion of the population voting by social class has moved from 75% in 1974 to 21% in 2010, according to MORI. Gone in the era when, as political scientist Peter Pulzer put it in 1967, class was "the basis of British party policies". There is no longer a gigantic bloc of working-class voters waiting to welcome Labour back - or, if there is, then they certainly do not see themselves as such.

In the socio-political research that I and colleagues do, we no longer view issues through the prism of class at all, but instead focus on people's core motivations, using an approach known as Values Modes. This segments the population into three groups: Pioneers, Prospectors and Settlers. Pioneers are inner-directed and concerned about fairness. Prospectors are optimistic, competitive and esteem-driven. Settlers are socially conservative and worried about external threats to resources.

Settlers are, almost exclusively, the group voting for Ukip. They tend to be older and from non-metropolitan areas, and they feel left behind: by London, by politics and by globalisation. They are not a particularly materialistic or acquisitive group, but rather they are focused on identity and belonging - meaning they are more driven by the social dimensions of policies than by the economic ones. They do not define themselves by political ideology or social class, but instead think in terms of their family and their street. They are frustrated by feeling ignored, and are resistant to top-down, statistics-led communications, telling them they are wrong on issues like immigration.

Ukip has - inadvertently or by design - succeeded in striking a chord with these Settler values in a way the other parties have not. Having shrunk significantly during the boom years, the Settler group has grown since the 2008 economic crisis, as resources have become scarcer - now making up 31% of the population. And thus, Farage's constituency has expanded, almost without him needing to do anything.

This shift is profound, and it will not be redressed by falling back on class orthodoxy. Instead, people interested in genuinely understanding Ukip must go beyond talking about class, and look at how voters think and feel.

Only by doing this can we recognise the realignment that has taken place, and start to understand the core values driving 21st Century voting patterns.