Evidence Based Policy has quickly established itself amongst Liberals as the new basis for politics. Its proponents, like drug expert David Nutt, the ‘Spirit Level’s Richard Wilkinson and Kate Pickett, and Ben Goldacre, argue that just as evidence-based-medicine establish the efficacy of drugs by measuring their impact against a double-blind, randomly selected control group, the same methodology can and should determine the policies politicians adopt. There are no problems with translating the scientific method to the social world.
To quote Goldacre, ‘A judge making a decision on a criminal’s sentence is in the exact same position as a doctor making a decision on a patient’s treatment’. By selecting two groups and giving different treatments, and ‘measure how well they’re doing a year or so later, you instantly discover which intervention is best’. Instead, we have a situation whereby venal politicians decide policy by ‘using their special magic politician beam’. To proponents, only ‘total cretins’ (or Daily Mail readers) could possibly have anything bad to say about EBP. Unfortunately, being a life-long Guardianista who believes strongly in drug and penal reform, as well as the moral necessity of creating a more equal society, and who passionately believes in the Enlightenment, rationalism and the scientific endeavor, I must be a cretin.
EBP problems are quite simply, vast, and they have to acknowledged. Firstly, even assuming it can determine the efficacy of certain policies, it cannot determine which values society should adopt - and therefore which programmes to implement. Simply reading data cannot give you a belief system.
Secondly, it assumes the effects of policies are wholly measurable and autonomous. The Welsh Assembly for example, has spent considerable money trying to engender an entrepreneurial culture of self-reliant, self-motivated risk-taking, in order to encourage small business start-ups. It has also spent considerable money trying to engender a healthy-lifestyle culture where people listen and obey government advise on how to eat, how to live and how to avoid taking risks. Both programmes could be effective and desirable, yet its is not too unrealistic to assert them incompatible - even if such incompatibility could be measured ‘in a year or so’. In the social world - far more than the natural - the causal consequences of actions are far too great and far too broad to fully calculate. After all, a random accident to just one of your ancestors thousands of years ago would have meant that you, and all your decedents, would never exist. EBP is built on overlooking this fact, because it begins to unravel once it’s accepted.
Contrary to Goldacre, Judges and Doctors are not in exactly the same position. If I’m seriously ill, and there’s a treatment with a 40% chance of helping, 40% of being neutral, and 20% of being detrimental, I’ll take the treatment, just as I take all the risk. A judge sentencing a murderer to a shorter sentance, as there’s a 40% that he’ll be better adjusted to society than being let out later, 40% of being no different, and a mere 20% chance of murdering again has to take a different perspective, as unlike the patient, the negative risk is borne not by the murder but by society. Both the ethics and ontology are very different.
As well as being impossible to measure all the policy’s consequences, they can work one way for ‘a year or so’ (to use Goldacre’s timeline) and an opposite later. Torture and internment can give you short term success in combating terrorism in the first couple of years, but can be counter-productive in motivating more terrorists - but such consideration must fall out of EBP practitioners, as they cannot be counted, or don’t exist in the ‘first year or so’.
Another problem with EBP is that it not only can confuse correlation with causation, but frequently does confuse necessary cause with sufficient cause and contributory cause. Take David Nutt’s examination of the social cost of alcohol. He argues (or assumes) that alcohol causes anti-social behaviour. But does it? I’ve been drunk many times, but never been violent; neither have my friends and family. Am I and they genetic freaks? Or, is it that alcohol uninhibits people who are violent or anti-social? If the latter, then the ultimate ‘cause’ is not alcohol, but issues of society, economics, personal psychology, morality, gender, amongst other things. Reducing anti-social behaviour by raising the price of alcohol not only masks the broader social problem, but punishes those of us who are part of the lower orders but are perfectly capable of drinking without being violent. The underlying class politics can be seen by Simon Jenkins’ (a sudden convert to scientific politics) - arguing that drugs (including alcohol) should be available, but only to the responsible middle-classes.
EBP does provide a useful snapshot in time. However, societies change. In fact, that’s all they ever do. Interventions that work in one cultural setting and time, may not work in another - but EBP’s authority rests on assuming they do not. Nutt’s assessment of the relative social damage of alcohol versus heroin is based on what society and culture is like right now. But social policy must be about what will society and culture be like in the future, (a future incidentally, that Nutt wants to simultaneously shape and be objective and disinterested about) and how it would change once/if drugs became legalised. Science can predict the future very well in the natural world - that’s mainly what gives it its power - but it simply cannot do so in the social world. In an interview published today, he conceeds that all he's done is 'start somewhere' and that 'there are so many unknowns' [about the the future], yet he remains adamant that 'the science is clear'. It patently is not. This undermines the entire basis of EBP.
Additionally, those subjected to EBP are not molecules, but intelligent individuals with motivations - good and bad. Lets say a hospital manager wants to improve patient care by reducing waiting times. They set a limit of four hours and see what happens. It’s clear that after ‘a year or so’, they come down. Evidence-Based-Management at its best! However, what actually happens is that instead of having a few patients wait five hours and most treated in two, almost all wait three hours, fifty minutes. Not only that, but health professionals now resent having their medical judgement overruled by arbitrary management targets. They gradually become embittered with the working culture, and only work what they must, not what they feel they should. Is this progress? Well, you can’t measure the causes of culture in EBP, so it must be. Certainly no hospital manager should consider using their ‘special magic manager’s beam’ and predict these consequences and factor them in accordingly.
Other problems with EBP is their underlying assumptions. There’s the inductive fallacy that the past can tell you what will happen in the future (does nothing happen for the first time? Just ask Bertrand Russell’s chicken), and there’s the assumption that human society conforms to the Gaussian bell-curve. However, as Nicholas Nassim Taleb has pointed out, we increasingly live in an age defined by low-probability, high-impact events - such as 9/11, the ‘blue-skies-thinking’ invention of the internet, and the credit crunch. In fact, the financial industry - packed with maths and physics graduates who based their models on laboriously acquired and analysed empirical evidence - determined that the chance of a crash was infinitesimally small. In contrast, the anti-gaussian epistemologists (like Taleb) and the anti-positivist socio-economists (like the Guardian’s Larry Elliott) saw it as inevitable. Crucially, they could only call it right precisely because they knew economics couldn’t be treated as a science, not least as it created false confidence and subsequent speculative bubbles. In short, it is not too much to say that evidence-based-banking caused the credit crunch.
However, there is a political danger to EBP too. Ultimately, it argues that the role of democratically elected politicians should be little other than subservient conduits to the advice given by the scientific priesthood. Even if their advice would be infallible (and for the arguments outlined above, we can be sure that it would not), there’s a genuine antidemocratic threat posed - nor is it historically unprecedented. Liberal scientists in the Weimar Republic regarded politicians as venal and ideologically blinkered, as they wouldn’t do as the scientists told. When the political crisis came, political liberal like Hans Morgenthau argued that by focusing on value-free objectivity, the supposedly liberal middle class offered little or no ideological opposition to the extremists. Lest people think the example is unrepeatable, the scientist and Liberal writer Martin Robbins argued in the Guardian that he only opposes torture as there’s no evidence of its effectiveness - meaning if it was proven, he would support it. (Naturally, no mention was made about the impact of torture on creating future terrorists. As that is unmeasurable, it must be barred from consideration.) It is deeply worrying when liberals abandon human rights so cheaply.
Politics is - or should be - about more than implementing the findings of a technocratic, unelected, scientific elite. It is about values, and competing visions of what society should be and can be. EBP not only cannot offer that, but threatens to actively undermines it.
EBP advocates evangelise as if no-one had thought of scientific politics before. In fact, the Soviet Union was practically built on bringing the evidence-based ‘scientific management’ of Frederick Taylor to the whole of society. It collapsed in no small part by the inherent impossibility of such a project. Not only that, but most twentieth century epistemological debates have centred on asking whether scientific methods can be applied to the social world. I have yet to see any EBP evangelist reflect on this historical examples of scientific politics, or make any serious attempt to deal with the critiques of scientific politics by Weber, Popper, Hayek, Morgenthau or Orwell, (let alone Adorno, Foucault and Gadamer) even though they were writing for a layman’s audience and can be read in a matter of hours or even minutes.
Much of the renaissance of popular science has been based on highlighting the ignorance of quacks who make claims in areas they have little knowledge, and no desire to learn. Their success has bred confidence in EBP evangelists that democratic and ideological politics can be replaced by scientific expertise. However, in assuming that the methods of assessing the natural world can be replicated unproblematically in the social, whilst blatantly - at times, even proudly - remaining ignorant of the intense philosophy of social science debates of the past century, they run the risk of becoming the same cargo-cult, blinkered pseudoscientists that they despise.
All this is definitely not to say that EBP is worthless. Seeking to gain some evidence of effectiveness is crucial. Evidence must be considered, and randomised, double-blind trials are an excellent way of gathering it. Nevertheless, EBP is inherently and inescapably flawed and partial, not least because it deals with a future that no-one - scientists included - cannot predict. Part of the problem here is the word ‘based’. Evidence Based Medicine is in practice, Evidence Determined Medicine, whereas EBP should in practice be Evidence Informed Policy.
Good government and healthy politics therefore demands a combination of empirical evidence and ‘special magic politicians beam’, or to use more traditional terms, vision, values and ideology. The combination is always complex, fraught and irreconcilable. But history, logic and an open society all demand that neither deserves to triumph fully.
