Last month, Transparency International released its annual Corruption Perceptions Index (CPI). As usual, the release of the CPI has generated widespread discussion and analysis. Previous GAB posts have discussed many of the benefits and challenges of the CPI, with particular attention to the validity of the measurement and the flagrant misreporting of its results. The release of this year’s CPI, and all the media attention it has received, provides an occasion to revisit important questions about how the CPI should and should not be used by researchers, policymakers, and others.
As past posts have discussed, it’s a mistake to focus on the change in each country’s CPI score from the previous year. These changes are often due to changes in the sources used to calculate the score, and most of these changes are not statistically meaningful. As a quick check, I compared the confidence intervals for the 2015 and 2016 CPIs and found that, for each country included in both years, the confidence intervals overlap. (While this doesn’t rule out the possibility of statistically significant changes for some countries, it suggests that a more rigorous statistical test is required to see if the changes are meaningful.) Moreover, even though a few changes each year usually pass the conventional thresholds for statistical significance, with 176 countries in the data, we should expect some of them to exhibit statistical significance, even if in fact all changes are driven by random error. Nevertheless, international newspapers have already begun analyses that compare annual rankings, with headlines such as “Pakistan’s score improves on Corruption Perception Index 2016” from The News International, and “Demonetisation effect? Corruption index ranking improves but a long way to go” from the Hidustan Times. Alas, Transparency International sometimes seems to encourage this style of reporting, both by showing the CPI annual results in a table, and with language such as “more countries declined than improved in this year’s results.” After all, “no change” is no headline.
Although certain uses of the CPI are inappropriate, such as comparing each country’s movement from one year to the next, this does not mean that the CPI is not useful. Indeed, some critics have the unfortunate tendency to dismiss the CPI out of hand, often emphasizing that corruption perceptions are not the same as corruption reality. That is certainly true—TI goes out of its way to emphasize this point with each release of a new CPI— but there are at least two reasons why measuring corruption perceptions is valuable: Continue reading