Last week, I used Professor Michael Johnston’s recent post on the methodological and conceptual problems with national-level perceived corruption indicators as an opportunity to respond to some common criticisms of research that relies on these indicators. In particular, I have frequently heard (and interpreted Professor Johnston as advancing) two related criticisms: (1) composite indicators of “corruption” are inherently flawed because “corruption” is a multifaceted phenomenon, comprised of a range of diverse activities that cannot be compared on the same scale, let alone aggregated into a single metric; and (2) corruption is sufficiently diverse within a single country that it is inappropriate to offer a national-level summary statistic for corruption. (These points are related but separate: One could believe that corruption is a sufficiently coherent concept that one can sensibly talk about the level of “corruption,” but still object to attempting to represent an entire country’s corruption level with a single number; one could also endorse the idea that national-level summary statistics can be useful and appropriate, even when there’s a lot of intra-country variation, but still object to the idea that “corruption” is a sufficiently coherent phenomenon that one can capture different sorts of corruption on the same scale.) For the reasons I laid out in my original post, while I share some of the concerns about over-reliance on national-level perceived corruption indicators, I think these critiques—if understood as fundamental conceptual objections—are misguided. Most of the measures and proxies we use in studying social phenomena aggregate distinct phenomena, and in this regard (perceived) corruption is no different from war, wealth, cancer, or any number of other objects of study.

Professor Johnston has written a nuanced, thoughtful reply (with a terrific title, “1.39 Cheers for Quantitative Analysis”). It is clear that he and I basically agree on many of the most fundamental points. Still, I think there are still a few places where I might respectfully disagree with his position. I realize that this back-and-forth might start to seem a little arcane, but since so much corruption research uses aggregate measures like the Corruption Perceptions Index (CPI), and since criticisms of these measures are likewise so common, I thought that perhaps one more round on this might not be a bad idea.

Let me address the two main lines of criticism noted above, and then make some more general observations. Continue reading →