Evidence

Yesterday I went to a Policy Exchange event with Theresa May and Professor Matt Qvortrop discussing the merits of referendums, or Citizen's Initiatives as he called them, and their capacity to reengage citizens with politics.

He was arguing for, as he described it, a "supply-side democracy". This is the idea that supplies of democracy create their own demand. If involved in decision making more often people will become more engaged in the political process. This seems fairly cogent. However, Qvortrup ruined his case with the methodology behind his paper The Voting Happiness Index which we were all given a copy of before the event.

I took a flick through it before the speech started and was alarmed. The report is filled with charts illustrating a correlation between various 'goods' from happiness to green politics to voter turnout. He used this as evidence that voter turnout led to all these good things.

Now, if you're going to argue that various good things are caused by voter initiatives a correlation is of very limited use. All a correlation suggests is some kind of relationship. It seems just as plausible that high democratic involvement, caused by some factor not in the study such as historical differences, was the driving force behind not just the other good things about Citizen's Initiative states but also the existence of Citizen's Initiatives in the first place. A state with a more democratically active citizenry is, after all, far more likely to be able to justify running referenda. This same taste for civic participation could also be the cause of all the other differences between Citizen's Initiative and non-Citizen's Initiative states. High voter engagement could be caused by any number of factors not included in the study.

In short, Qvortrup's study has no way to tell which factor is cause and which effect. It can do almost nothing to establish whether an increase in the amount, supply of voting makes any difference at all to the success of a democracy.

Also, he used happiness statistics and quoted Layard on how great they are. They aren't. Happiness statistics are shoddy and unreliable.

Finally, his evidence had clearly been cherry-picked. The Citizen's Initiative states may have been greener but I'm willing to put money on them also having been less fiscally disciplined.

I asked Qvortrup about these issues and he smiled genially and said he was just making the case for his side, of course he chose data that would strengthen his case. My heart sank a little. Partly because I was deeply aware that Qvortrup wasn't really doing anything particularly wrong. He hadn't claimed his data should change the world and the methodological weaknesses of his study pale in comparison with some other, far more prestigious, works.

I emerged from the talk neither more or less convinced of the case for making referendums a regular part of our political landscape. Instead, I came away with a renewed sense of how careful you need to be when told "studies show that..."