More on the “News from Nowhere” Problem in Anticorruption Research

One of my all-time favorite academic papers — which should be required reading not only for those who work on anticorruption, but on any topic where people casually throw around statistics — is Marc Galanter‘s 1993 article News from Nowhere: The Debased Debate on Civil Justice. Professor Galanter’s paper doesn’t have anything directly to do with international corruption. Rather, he sets out to debunk a series of widely-held but mostly-false beliefs about civil litigation in the United States, and in the process he traces the origins of many of the statistics often cited in debates about that topic. He finds that many of these statistics come from, well, nowhere. Here’s my favorite example: Around the time Professor Galanter was writing, it was common to hear claims that the civil justice system costs $80 billion in direct litigation costs; indeed, that figure appeared in an official report from the President’s Council on Competitiveness. The report’s only source for that estimate, however, was an article in Forbes; Forbes, in turn, had drawn the figure from a 1988 book by Peter Huber. But Huber himself hadn’t done any direct research on the costs of the system. Rather, Huber’s only source for the $80 billion figure was an article in Chief Executive magazine, which reported that at a roundtable discussion, a CEO claimed that “it’s estimated” (he didn’t say by whom) that insurance liability costs industry $80 billion per year. So: A CEO throws out a number at a roundtable discussion, without a source, it gets quoted in a non-scholarly magazine, repeated (and thus “laundered”) in what appears to be a serious book, and then picked up in the popular press and official government reports as an important and troubling truth about the out-of-control costs of the US civil justice system.

I thought about Galanter’s book the other day when I was reading the Poznan Declaration on “Whole-of-University Promotion of Social Capital, Health, and Development.” The Declaration itself is about getting universities to commit to integrating anticorruption and ethics into their programs; I may have something to say about the substance of the declaration itself in a later post. But the following assertion in the Declaration caught my eye: “Despite the relative widespread implementations of anti-corruption reforms and institutional solutions, no more than 21 countries have enjoyed a significant decrease in corruption levels since 1996, while at the same time 27 countries have become worse off.” Wow, I thought, that seems awfully precise, and if it’s true it’s very troubling. Despite the fact that I spend a fair amount of time reading about the comparative study of corruption, that statistic is news to me. It turns out, though, that it’s news from nowhere. Continue reading

Let’s Create Sub-National Corruption Perception Indexes for the BRICS

For all their flaws, the major cross-country corruption indexes—Transparency International’s Corruption Perceptions Index (CPI), the World Bank Institute’s Worldwide Governance Indicators (WGI), and the like—have been quite useful, both for research (at least when used appropriately) and for advocacy.  But one important limitation of these datasets is that by focusing on corruption (or perceived corruption) at the country level, they may obscure the fact that there can be substantial within-country variation in the level of (perceived) corruption.  This variation may occur across government institutions—the same country may have quite different degrees of corruption in the health sector, the police force, the judiciary, customs, etc.  More pertinent here, there may also be significant heterogeneity across regions, particularly in large countries with substantial political decentralization.  Indeed, numerous studies have exploited within-country regional variation in corruption levels to test various hypotheses about corruption’s causes and consequences; such studies include research on Italy, Russia, China, the Philippines, and the United States, among others.  But these studies typically make use of particular data sets that are not reproduced year-to-year.

As we’re starting to see rapidly diminishing returns from the major cross-country corruption datasets, it is high time for those organizations with the resources and capacity to compile information on corruption perceptions on an ongoing basis to turn their focus to within-country regional variation in corruption.  I propose the creation of a sub-national corruption perceptions index (snCPI), starting with the so-called BRICS countries (Brazil, Russia, India, China, and South Africa), which would gather and compile data (primarily perception-based data, perhaps supplemented with more objective data when available) on perceived corruption levels within the major sub-national units (states/provinces, autonomous regions, and municipalities) within each of those countries.

Continue reading

Objective Validation of Subjective Corruption Perceptions?

As discussed on this blog and elsewhere, one of the big concerns about the most popular cross-country datasets on corruption (the Transparency International Corruption Perceptions Index (CPI), the World Bank Institute’s Worldwide Governance Indicators (WGI), etc.) is that they are based (largely or entirely) on perceptions of corruption. As Rick noted in a recent post, and as the critical literature has pointed out ad nauseam, perceptions, while perhaps important in their own right, are not necessarily based in reality. Indeed, some recent research (including, but certainly not limited to, nice papers by Claudio Weber Abramo, by Mireille Razafindrakoto and Francois Rouband, and by Richard Rose and William Mishler) indicates that national corruption perceptions are only weakly correlated with survey results asking about individuals’ personal experience with bribery. This raises serious questions about whether the perception-based indicators are useful either for general assessment or for testing hypotheses about the causes or consequences of corruption.

But might there be more objective measures that could be used to assess whether the corruption perceptions indices are picking up something real? Off the top of my head, I can think of four quite clever recent papers that demonstrate a strong correlation between a subjective corruption perception index and some more objective measure of dishonest behavior. I’m sure there are more, but let me note the four examples that I can think of, and then say a bit on what this might mean for the use of perception-based indicators in empirical corruption research.

Continue reading

Does the Social Value of Corruption Indicators Depend Solely on Their Accuracy?

We’ve had a series of posts this week (from Michael, Rick, and Addar) about the vexed question of how to measure corruption–including controversies over whether the popular perception-based measures, like Transparency International’s Corruption Perceptions Index (CPI), do so accurately enough to be useful proxies.

But in addition to that discussion — and picking up on something Rick touched on in the latter half of his post — I think it may be worth raising the questions whether: (1) something like the CPI might have desirable effects even if the CPI score is not a particularly good indicator of true corruption, and (2) whether the CPI might have bad effects even if it’s actually quite an accurate measure. To be clear, my very strong predispositions are that we should try to produce and publicize accurate measures of corruption. But it’s at least worth thinking about the possibility that the social value of an indicator may not depend entirely on the accuracy of that indicator–and about what implications might follow from that observation. Continue reading