There are, says Prof Tim Spector in The Diet Myth, more than 30,000 books out there on what to eat, offering a plethora of different diets, some sensible and some downright dangerous, and between them pulling in all possible directions (no sugar, no fat (or more!), less protein (or more!), more fruit (or less!), and so ad infinitum). Most people in the western world seem to have tried some kind of diet at some time or another though mostly to lose weight rather than to stay healthy. Most dieters do lose a few pounds at first but most soon give up through boredom or because they don't feel too good. Few diets serve their followers well in the long haul. Yet many if not most of these excursions are based on some kind of science - refereed papers - somewhere along the line, and science tells us what is true, does it not? So what's gone wrong?
Well, for the last half century nutritional science has been in flux -- one damn' idea after another. But within the melee there is revelation, too. For now it seems that bodies are not simply machines, even complex machines, as Rene Descartes told us nearly 400 years ago (and scientists have been eager to believe). Biology, including nutrition, cannot be "reduced" to chemistry, but has its own rules. Even more to the point, the insights, the "truth" of science is not absolute, as has been the assumption, but is always partial and provisional; and life is not a "problem" to be solved, but is and will always be a mystery, which can and should be probed forever but can never be exhaustively understood. Nutritional science, in short - and all science - is in the throes of what Thomas Kuhn in the 1960s called a "paradigm shift"; a change of worldview. The shift is a very good thing and well overdue. It's time science was knocked off its high and authoritarian horse. But seismic shifts come with a price. In nutritional science, the flurry of diets, pulling every which way, is the fall-out.
It's clear for a start that nutrition cannot be studied in isolation. The gut and its contents at any one time interact, both ways, with all other body systems. Nutritional response is a matter not just of gut physiology and biochemistry but of genetics, endocrinology, immunology, neurology and psychology - all shaped, of course, as all biological systems are, by the vagaries and diversions of evolutionary history (for as the great American Theodosius Dobzhansky famously commented, "nothing in biology makes sense except in the light of evolution").
All this means that the Cartesian model of the body-as-machine is most inappropriate. Machines do what they are told - press the button and out comes the additive-packed donut - but the body's response to stimulus is always conditional; it depends on recent experience and past history, genetic and otherwise, and on what else is going on. In other words, living creatures (even those without brains, like oak trees) respond intelligently to stimuli. An organism can be coaxed, even orchestrated, but so long as it remains intact it cannot simply be bossed around.
A series of particular observations have contributed to the change of mind. First, nutritionists have recognised a new class of what some call "cryptonutrients"; agents - probably many thousands of them - produced by plants, fungi, and microbes in quantities often too small to register, but ever-present, which sometimes poison us but often serve as nutrients or tonics. The food and pharmaceutical industries have cottoned on to cryptonutrients (and call them "nutraceuticals" or "functional foods") and now for example offer margarine with added plant sterols (which are said to lower blood cholesterol) and yoghurts full of "friendly bacteria", which are supposed in a broad-brush way to sort us out, for microbes between them are the world's greatest pharmacists.
More broadly, it's clear that gut microbes, once seen merely as hangers-on and spivs, though always liable to turn rogue and strike us down, in fact run the show. How we respond to food, whether it does us good or makes us ill, depends critically on which and how many bacteria and archaeans we are harbouring. We pick up our allotted stash from our mothers at birth (Caesarean birth may screw this up) and some at least stay with us for life, though we acquire many more along the way. We have to treat our microbes well - so good dieting becomes a matter not simply of biochemistry but of ecology. Ecology is an exercise in complexity. The relationship between cause and effect is never press-the-lever-and-take-your-choice. It is non-linear.
The lore of the past century too has been that our genes, now known as DNA, run the body from their nuclear stronghold, like the high command; giving orders but not so good at listening. Now the emerging science of epigenetics is showing that the genes are in constant dialogue with the cell around them and hence with the body as a whole and so with the outside world, and the way genes work or whether they are allowed to work at all depends very much on what goes on around them - and indeed, on what happened to their host's mothers and grandmothers, for epigenetic effects may pass from generation to generation.
More broadly still, it's clear that Donald Rumsfeld's adage applies: we are faced not only with unknowns but with unknown unknowns, things we didn't even know we didn't know. Or, to borrow an idea from the dramatist/philosopher Gabriel Marcel, life is not a problem that can be solved, if we do enough research. It is a mystery to be probed, and the more we probe the more we will find that there is to find out. Final solutions are not possible even in theory and algorithms, including diets, should always be applied with fingers crossed (or not at all).
The bevy of diets pulling this way and that results in large part from desperation on the one hand - the misguided impulse to find magic bullets for all our ills; and over-confident science - the tendency to leap from intriguing observation to dogma to recommendation without properly appreciating the scale or even the nature of the problem in hand. Science in general even at its best does not deliver certainties, and certainly cannot deal in unequivocal truth.
That is a most satisfying note to finish on; science as the servant of humankind and of the biosphere; a wondrous source of insight, showing us (up to point!) how marvellous life and the universe really are; very much a part of what we need to know but not, absolutely not, the font of all wisdom. I am sticking to my Grannie's advice: "A little of what you fancy does you good". Scientists should always ask (though they very rarely do) whether they in practice improve on folklore.
Colin Tudge's latest book, Six Steps Back to the Land, is available from Green books. A longer version of this article can be found in the website of the College for Real Farming and Food Culture . The paradigm shift in nutritional theory will be discussed at next year's Oxford Real Farming Conference (Jan 4-5 2017)