This Year, Let’s All Resist the Temptation to Emphasize Changes in Individual Country’s CPI Scores!

Later this week (if I’m not mistaken, a couple of days from today) Transparency International (TI) will publish its annual Corruption Perceptions Index (CPI), together with some press materials and additional discussions. And if this year is like previous years, many media outlets — and TI itself — will make much of how individual countries’ scores and rankings have changed from the previous year. Often these discussions will be situated into some narrative (usually along the lines of, “Country X’s anticorruption efforts are failing, as we can tell from its declining score”). In fact, sometimes politicians and activists will point to their country’s score changes as evidence on the question whether they are making progress on the fight against corruption.

This comparison of annual CPI scores for individual countries is, with vanishingly few exceptions, a pointless, misleading, intellectually bankrupt exercise, for reasons that I’ve tried to explain pretty much every year for the last seven years. See here, here, here, here, here, here, here, and here. To be clear, I’m a fan of the CPI and will continue to defend it as a worthwhile measurement exercise, despite its flaws. And many of the folks on the TI research team who work very hard on this index are smart, serious people who are doing their best. Indeed, if you know where to look, you can sometimes find TI research documents on the CPI that include appropriate caveats. But TI’s press releases and public comments, and most of the media commentary on the CPI, continue to treat individual changes in each country’s score as some kind of meaningful indicator.

This year, I’m going to try something new. Instead of waiting until after the CPI is published, and then sitting back in my (metaphorical) armchair in the Ivory Tower and hurling criticisms at those who portray year-to-year changes in individual countries’ CPI scores as meaningful, I’m going to try raising this issue before the CPI is published, in the hopes that this might have more of an impact in how the CPI numbers are presented, especially by the folks at TI. (And I know some of you read this blog!!!) It’s not too late! Please please please go over your press release and other materials and make sure you’re not presenting your (very important!) work as telling us anything interesting or useful about which individual counties are getting better or worse as compared to last year (or the last few years). Please please please emphasize that the CPI is not meant to be used as an indicator of policy success or failure. Please please please, at the very least, make sure that you emphasize the uncertainty (that is, the “noisiness”) of the perception estimates (which is not the same as the point that perceptions are different from reality, which TI already emphasizes), and for goodness’ sake, don’t emphasize score changes that your own data indicates are not statistically significant at conventional levels.

And in case any of you folks in the media happen to be reading this blog, you can do better too! The CPI is a great “hook” for discussing corruption-related issues in your country, but you do your readers a disservice if you cover the CPI as if it’s a league table, or try to construct a narrative around random noise.

(Oh, by the way, all of the above exhortations are premised on the validity of my critique of year-to-year country CPI comparisons. If anyone out there thinks that critique is misguided, I would also welcome a substantive rebuttal. I’m not going to restate all the elements of my critique here; anyone who is interested can click on the links above and read my posts from previous years.)

Let’s see if this preemptive strike is any more successful than past years’ after-the-fact criticisms…

1 thought on “This Year, Let’s All Resist the Temptation to Emphasize Changes in Individual Country’s CPI Scores!

  1. I first came across CPI in 2005 while attending The Economics of Corruption at University of Passau, Germany annually organized by Prof Graff Lambsdroff, the inventor of CPI. I find the things bit technical but, at that time, ensuing debate was how to make one year CPI comparable to other year. Due to difference in data sources, we were asked not to compare one year CPI with that of another year unless data sources remain same. I suppose this issue has now been corrected, ie, CPI annual data are comparable since 2013 or 2014. Prof Lambsdroff is no more associated with CPI production for TI. Another debate about CPI was to publish data for, say every, five or ten years, instead of annual publication. I suppose this is important debate as corruption cannot be controlled overnight. The third debate that ensued was that only countries at the middle of the CPI ranking were being stressed. Those at the top and those at the bottom did not care about CPI data. There are hosts of other indicators reflecting the state of corruption – eg Control of Corruption Indicator for World Governance Indicators, Bribe Payers Index, Global Corruption Barometer, Public Integrity Index (University of Gottenberg, Sweden) etc. etc. There is a need for collating and coordinating proliferation of data and indicators related to corruption. Obviously, there are flaws with all indicators, but what the beauty is that: In order to depoliticize corruption issue, we need data on corruption and this is very hard to come by.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.