Re-Upping Last Year’s Post on How (Not) to Cover the About-To-Be-Released CPI

Later this week (tomorrow, I believe) Transparency International (TI) will release the newest version of its annual Corruption Perceptions Index (CPI). I was going to do a post on this, but I realized as I started writing it that what I had to say was virtually identical to what I wrote at this time list year. So I’m just going to paste below the text of last year’s post, while noting my sincere hope that by next year, or at least within a few years, this will no longer be relevant:

Later this week (if I’m not mistaken, a couple of days from today) Transparency International (TI) will publish its annual Corruption Perceptions Index (CPI), together with some press materials and additional discussions. And if this year is like previous years, many media outlets — and TI itself — will make much of how individual countries’ scores and rankings have changed from the previous year. Often these discussions will be situated into some narrative (usually along the lines of, “Country X’s anticorruption efforts are failing, as we can tell from its declining score”). In fact, sometimes politicians and activists will point to their country’s score changes as evidence on the question whether they are making progress on the fight against corruption.

This comparison of annual CPI scores for individual countries is, with vanishingly few exceptions, a pointless, misleading, intellectually bankrupt exercise, for reasons that I’ve tried to explain pretty much every year for the last seven years. See here, here, here, here, here, here, here, and here. To be clear, I’m a fan of the CPI and will continue to defend it as a worthwhile measurement exercise, despite its flaws. And many of the folks on the TI research team who work very hard on this index are smart, serious people who are doing their best. Indeed, if you know where to look, you can sometimes find TI research documents on the CPI that include appropriate caveats. But TI’s press releases and public comments, and most of the media commentary on the CPI, continue to treat individual changes in each country’s score as some kind of meaningful indicator.

This year, I’m going to try something new. Instead of waiting until after the CPI is published, and then sitting back in my (metaphorical) armchair in the Ivory Tower and hurling criticisms at those who portray year-to-year changes in individual countries’ CPI scores as meaningful, I’m going to try raising this issue before the CPI is published, in the hopes that this might have more of an impact in how the CPI numbers are presented, especially by the folks at TI. (And I know some of you read this blog!!!) It’s not too late! Please please please go over your press release and other materials and make sure you’re not presenting your (very important!) work as telling us anything interesting or useful about which individual counties are getting better or worse as compared to last year (or the last few years). Please please please emphasize that the CPI is not meant to be used as an indicator of policy success or failure. Please please please, at the very least, make sure that you emphasize the uncertainty (that is, the “noisiness”) of the perception estimates (which is not the same as the point that perceptions are different from reality, which TI already emphasizes), and for goodness’ sake, don’t emphasize score changes that your own data indicates are not statistically significant at conventional levels.

And in case any of you folks in the media happen to be reading this blog, you can do better too! The CPI is a great “hook” for discussing corruption-related issues in your country, but you do your readers a disservice if you cover the CPI as if it’s a league table, or try to construct a narrative around random noise.

(Oh, by the way, all of the above exhortations are premised on the validity of my critique of year-to-year country CPI comparisons. If anyone out there thinks that critique is misguided, I would also welcome a substantive rebuttal. I’m not going to restate all the elements of my critique here; anyone who is interested can click on the links above and read my posts from previous years.)

Let’s see if this preemptive strike is any more successful than past years’ after-the-fact criticisms…

3 thoughts on “Re-Upping Last Year’s Post on How (Not) to Cover the About-To-Be-Released CPI

  1. The critique is on target, but much too limited in my opinion, as the CPI, by ranking countries, also misleads in identifying corruption as a national-level problem rather than an intrinsically global problem. For understanding the most consequential corruption in today’s world, the indexes from the Tax Justice Network are far more important in tracking the most pervasive grand corruption. That corruption is rather in yet another country that is both nowhere and everywhere, where the wealth of kleptocrats, plutocrats, multinational corporations, and simply HNWIs https://www.investopedia.com/terms/h/hnwi.asp keep much of their wealth and make a large proportion of their transactions.

    See https://fsi.taxjustice.net/en/ for the Foreign Secrecy Index and https://cthi.taxjustice.net/en/ for the Corporate Tax Haven Index.

    In the courses in methodology that I took in graduation school in Sociology, that would be a methodological error of mistaking the most appropriate unit of analysis to explain a social reality, in this case, corruption.

    Of course, one can debate the definition. But if money laundering and tax evasion are not included in “corruption,” they are still close relatives of corruption, enablers of corruption and just as or more damaging to global society than corruption measured within the boundaries of nation states.

Leave a Reply to AfricaFocus Bulletin (@africa_focus) Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.