Recently I learned that CDA Collaborative (a nonprofit organization that works on a variety of development and conflict-resolution projects) has launched a new blog on corruption. Though it’s a new platform, they already have a few of interesting posts up, and it’s worth a look.
While I’m always happy to advertise new platforms in the anticorruption blogosphere, in this post I mostly want to focus on the first entry in the CDA’s new blog, a post by Professor Michael Johnston entitled “Breaking Out of the Methodological Cage.” It’s basically a critique of the anticorruption research literature’s alleged (over-)reliance on quantitative methods, in particular cross-national regression analyses using country-level corruption indices (such at the Corruption Perceptions Index (CPI) or Worldwide Governance Indicators (WGI) graft index). There are some things in Professor Johnston’s post that I agree with, and much that I disagree with. I want to focus on one issue in particular: the question of the right unit of analysis, or level of aggregation, to use when attempting to measure corruption.
Professor Johnston has two related complaints (or maybe two variants on the same underlying complaint) regarding these national-level perceived corruption measures. First, he complains these “[o]ne dimensional indices tell us … that corruption is the same thing everywhere, varying only in amount[.]” In other words, corruption indices lump a whole bunch of disparate phenomena together under the same umbrella term “corruption,” ignoring the internal diversity of that category. Second, he contends that “relying … on country-level data is to assume that corruption is a national attribute, like GDP per capita” when in fact “corruption arises in highly specific processes, structural niches, and relationships.” Corruption, he explains, is not an attribute of countries, but of more specific contexts, involving “real people … in complex situations[.]”
Respectfully, I think that these points are either wrong or irrelevant, depending on how they are framed.
I want to focus on this issue because it seems to come up a lot. In a way I’m picking on Professor Johnston’s post not because his points are unusual, but rather because his post is but one particularly articulate example of a line of criticism that one hears over and over and over again—that one cannot measure corruption at the country level, because “corruption” is not a useful category given the great variety of “corrupt” activities, and that one cannot usefully measure any of these kinds of corruption at the country level, given the internal heterogeneity within countries (say, across regions or government departments) with respect to the prevalence and type of corruption.
The statements (1) that the term “corruption” is a broad general category, and (2) that there’s a lot of internal diversity within countries, are both true. It does not follow, however, that one can’t or shouldn’t ever use a broad measure of corruption, assessed at the country level. Sometimes you shouldn’t, but sometimes you should—it depends on the research question. Virtually all categories we use, particularly those we use to describe characteristics of countries, have the two features noted above; corruption is not unique in this regard, so pointing out these facts about corruption is not, by itself, all that useful.
I think Professor Johnston inadvertently illustrates something like the point I’m getting at in the passage I quoted earlier, where he says that corruption is not “a national attribute, like GDP per capita[.]” But why, exactly, does Professor Johnston think that GDP per capita is a “national attribute”? GDP per capita is a shorthand way of capturing the annual average value of goods and services produced by an individual in the country in question. It is a summary measure that flattens out the great diversity of human activity that may create economic value. One could say of “GDP per capita” almost exactly what Professor Johnston says about corruption: That although macro-level factors may affect opportunities and constraints for economically productive activities, “real people decide to take advantage of them … or not – in complex situations, defined by perceived alternatives and consequences”—and national-level indicators like GDP per capita “tell us little about those complexities.” Does this mean we shouldn’t use GDP per capita when researching causes and consequences of economic development? Well, maybe it does; there are lots of criticisms of the various ways in which GDP per capita is used as a summary statistic, for example its omission of other aspects of human welfare. But the point is that there’s nothing that makes GDP per capita, but not (perceived) corruption, a “national attribute.” What matters is whether that level of aggregation (and inevitable simplification) is useful for the research question at hand.
Perhaps another example can further illustrate the point. There are statistics out there on national-level cancer rates. These aggregated statistics mask variation in the type of cancer. For some purposes, aggregating all cancers into a single summary statistic is fine. In other cases, disaggregating by type of cancer (pancreas, liver, etc.) is vital. In other cases, further disaggregation is required—sometimes each patient must be treated as unique. And sometimes the question we’re asking really means that per capita cancer rates should be integrated into a higher-level category, such as overall disease rates, rather than treated as a separate category. So too with corruption. For some research purposes, it’s sensible and appropriate to use a national-level aggregate evaluation. For other purposes, it’s essential to disaggregate by type of corruption (bribery, embezzlement, nepotism, etc.), or by magnitude (grand vs. petty, for example), or by region or department or something else. Sometimes each individual instance must be analyzed as sui generis. And sometimes “corruption” is actually too narrow a category, and we should instead try to measure, for example, the overall level of economic crime, or failures of governance, or what have you.
Now, I’m not sure that Professor Johnston would necessarily disagree with that. It might be possible to read his post not as an argument that there’s some inherent reason why perceived corruption—unlike GDP or cancer rates—simply cannot be expressed in terms of a single national-level summary statistic. Maybe what he means is that for the research questions we do or should care about regarding the causes and consequences of corruption, identifying correlates of these summary measures is not useful. If that’s what he’s saying, then he and I have no conceptual or methodological disagreement, though I think he may be underestimating the usefulness of some cross-country correlational studies. But again, my beef isn’t with Professor Johnston specifically—his post just provided a useful opportunity for me to lay out my criticisms of a point that I’ve heard plenty of other smart people make.
The bottom line here is that simply pointing out that a national-level summary statistic, such as a country’s score on the CPI, masks both diversity within the category and the country, is not by itself a deep or devastating critique of research that uses those measures. Those points are true but obvious, and common to just about every national statistical measure we might use for anything. The question is, or should be, whether the research using measures like the CPI and WGI can shed light on important questions.