Later this week (if I’m not mistaken, a couple of days from today) Transparency International (TI) will publish its annual Corruption Perceptions Index (CPI), together with some press materials and additional discussions. And if this year is like previous years, many media outlets — and TI itself — will make much of how individual countries’ scores and rankings have changed from the previous year. Often these discussions will be situated into some narrative (usually along the lines of, “Country X’s anticorruption efforts are failing, as we can tell from its declining score”). In fact, sometimes politicians and activists will point to their country’s score changes as evidence on the question whether they are making progress on the fight against corruption.

This comparison of annual CPI scores for individual countries is, with vanishingly few exceptions, a pointless, misleading, intellectually bankrupt exercise, for reasons that I’ve tried to explain pretty much every year for the last seven years. See here, here, here, here, here, here, here, and here. To be clear, I’m a fan of the CPI and will continue to defend it as a worthwhile measurement exercise, despite its flaws. And many of the folks on the TI research team who work very hard on this index are smart, serious people who are doing their best. Indeed, if you know where to look, you can sometimes find TI research documents on the CPI that include appropriate caveats. But TI’s press releases and public comments, and most of the media commentary on the CPI, continue to treat individual changes in each country’s score as some kind of meaningful indicator.

This year, I’m going to try something new. Instead of waiting until after the CPI is published, and then sitting back in my (metaphorical) armchair in the Ivory Tower and hurling criticisms at those who portray year-to-year changes in individual countries’ CPI scores as meaningful, I’m going to try raising this issue before the CPI is published, in the hopes that this might have more of an impact in how the CPI numbers are presented, especially by the folks at TI. (And I know some of you read this blog!!!) It’s not too late! Please please please go over your press release and other materials and make sure you’re not presenting your (very important!) work as telling us anything interesting or useful about which individual counties are getting better or worse as compared to last year (or the last few years). Please please please emphasize that the CPI is not meant to be used as an indicator of policy success or failure. Please please please, at the very least, make sure that you emphasize the uncertainty (that is, the “noisiness”) of the perception estimates (which is not the same as the point that perceptions are different from reality, which TI already emphasizes), and for goodness’ sake, don’t emphasize score changes that your own data indicates are not statistically significant at conventional levels.

And in case any of you folks in the media happen to be reading this blog, you can do better too! The CPI is a great “hook” for discussing corruption-related issues in your country, but you do your readers a disservice if you cover the CPI as if it’s a league table, or try to construct a narrative around random noise.

(Oh, by the way, all of the above exhortations are premised on the validity of my critique of year-to-year country CPI comparisons. If anyone out there thinks that critique is misguided, I would also welcome a substantive rebuttal. I’m not going to restate all the elements of my critique here; anyone who is interested can click on the links above and read my posts from previous years.)

Let’s see if this preemptive strike is any more successful than past years’ after-the-fact criticisms…