Last month, Transparency International released its annual Corruption Perceptions Index (CPI). As usual, the release of the CPI has generated widespread discussion and analysis. Previous GAB posts have discussed many of the benefits and challenges of the CPI, with particular attention to the validity of the measurement and the flagrant misreporting of its results. The release of this year’s CPI, and all the media attention it has received, provides an occasion to revisit important questions about how the CPI should and should not be used by researchers, policymakers, and others.
As past posts have discussed, it’s a mistake to focus on the change in each country’s CPI score from the previous year. These changes are often due to changes in the sources used to calculate the score, and most of these changes are not statistically meaningful. As a quick check, I compared the confidence intervals for the 2015 and 2016 CPIs and found that, for each country included in both years, the confidence intervals overlap. (While this doesn’t rule out the possibility of statistically significant changes for some countries, it suggests that a more rigorous statistical test is required to see if the changes are meaningful.) Moreover, even though a few changes each year usually pass the conventional thresholds for statistical significance, with 176 countries in the data, we should expect some of them to exhibit statistical significance, even if in fact all changes are driven by random error. Nevertheless, international newspapers have already begun analyses that compare annual rankings, with headlines such as “Pakistan’s score improves on Corruption Perception Index 2016” from The News International, and “Demonetisation effect? Corruption index ranking improves but a long way to go” from the Hidustan Times. Alas, Transparency International sometimes seems to encourage this style of reporting, both by showing the CPI annual results in a table, and with language such as “more countries declined than improved in this year’s results.” After all, “no change” is no headline.
Although certain uses of the CPI are inappropriate, such as comparing each country’s movement from one year to the next, this does not mean that the CPI is not useful. Indeed, some critics have the unfortunate tendency to dismiss the CPI out of hand, often emphasizing that corruption perceptions are not the same as corruption reality. That is certainly true—TI goes out of its way to emphasize this point with each release of a new CPI— but there are at least two reasons why measuring corruption perceptions is valuable:
- First, corruption perceptions are a good proxy for real corruption. As is well known, corruption’s secretive nature stymies objective measurement, and corruption perceptions are a low-cost way to produce broad-stroke cross-country comparisons for use in research and policy design. Indeed, a prior post on this blog discusses research showing that the CPI has a reasonably strong correlation with other more objective measures of corruption, such as diplomatic parking tickets as a measure of corruption norms, and corporate tax evasion by foreign firms in the US. Other research provides further validation, including microeconomic evidence that corruption perceptions contain information about real corruption levels.
- Second, the perception of corruption may have serious consequences in its own right. Democracy relies on some amount of trust in the government, and on the voluntary law-abiding behavior of most citizens. When a government is perceived to be corrupt, we can expect that trust in government will diminish.
The CPI is a valuable measurement for understanding and combating corruption. Rather than attaching significance to annual changes in a country’s CPI value, we must use it as a more nuanced policy and research tool. Some ways in which the CPI could be used include the following:
- International organizations can refer to the CPI in a given year when evaluating risk levels for corruption in foreign projects. For private corporations, aid organizations, and governments operating abroad, corruption poses a threat to the success of international ventures. In development aid in particular, corruption can mean that well-intentioned resources instead bolster the regime of corrupt governments. The CPI can be useful in evaluating expected corruption levels to plan for potential issues and evaluate risk. For example, consider Namibia and Angola, two neighboring countries with similar GDP per capita and poverty rates. Namibia has a 2016 CPI value of 52 and rank of 53/176, while Angola has a CPI of 18 and a rank of 164/176. Operations in Angola need substantially more oversight than those in Namibia, which may not be apparent without use of the CPI.
- Comparisons of corruption perceptions between similar countries can provide some evidence of the effectiveness of the respective transparency and public integrity policies of those countries. Of course, true policy research requires careful social science to understand the casual relationships at hand. However, CPI differences can be used for exploratory hypothesis generation. In this vein, we ask: what is Namibia doing successfully that can inform policy decisions in Angola?
In sum, while headlines that focus on short-term shifts in CPI values should be treated with extreme caution— particularly when they fail to conduct statistical inference on the differences they report— the CPI, when used correctly, is a valuable tool for policymakers, social scientists, and anyone working in the international sphere.
GAB | The Global Anticorruption Blog 
I don’t know how anyone could disagree with your over-arching analysis and conclusion Jetson. Thanks for this contribution.
If there is anyone out there who now believes the risks of misusing the CPI outweighs the benefits please present your case as we all have more important issues to tend to both here at home and abroad.
Thank you for your analysis. Your examination into the confidence intervals is quite interesting. I agree that one should try to avoid simply comparing one year’s score to another as the methodology can change, along with other factors. I would highly encourage viewers to also read through the source description, methodology notes, and data sets available on Transparency International’s website while looking at a country’s rating. Some countries tend to score high on some surveys while low on others; some countries have several data sources, while others only have 3. Since the CPI is calculated using a simple average after data sources are standardized, it is worth looking into which surveys are contributing information and what types of questions they actually being asked. Overall, it is a valuable tool and certainly worth taking into consideration for policymakers and/or other relevant stakeholders.
Thanks for the post. I agree that the year-to-year comparison can be misleading. For example, in the Pakistan case you alluded to saw a 2 point rise warrant a positive news article, with no analysis of the underlying factors. That said, it would be interesting to see studies that rigorously analyze the statistical significance of the Index. It would be a good way of keeping TI in check too, since you mentioned how they encourage the hyperbolic headlines on occasions by way of their methodology.
I am hunting for such a posting from GAB to explain the controversies and analysis of CPI year-wise. We are really grateful to TI for providing the only global Corruption Index sheet. As I am least concerned about the method and analysis resource of this CPI calculation country-wise, it is very essential for study the gross economic as well as political situation as per development. The faults are must be shorted out for more scientific picture that is much expected for non-bias global report, for research and measure of anti-corruption efforts.
Sure, for earlier GAB postings discussing the concerns about year-to-year comparisons of individual countries’ CPI scores, see the following posts:
https://globalanticorruptionblog.com/2014/09/30/pre-2012-cpi-scores-cannot-be-compared-across-time-so-please-stop-doing-it/
https://globalanticorruptionblog.com/2014/10/07/post-2012-cpi-scores-cant-be-compared-across-time-either/
https://globalanticorruptionblog.com/2014/11/20/more-on-cpi-changes-over-time-or-not/
https://globalanticorruptionblog.com/2015/11/10/a-quick-partial-fix-for-the-cpi/
https://globalanticorruptionblog.com/2016/02/04/the-2015-cpi-and-year-to-year-changes-a-definite-improvement-but-problems-remain/
https://globalanticorruptionblog.com/2014/12/04/transparency-international-makes-its-data-less-transparent-why-ti-should-be-ashamed-of-its-2014-cpi-report/
https://globalanticorruptionblog.com/2015/02/17/the-2014-cpi-data-demonstrates-why-even-post-2012-cpi-scores-cannot-be-compared-over-time/
Jetson, your point about “perception of corruption” being a meaningful variable in-and-of itself is an important one. To the extent voters are bringing concerns about corruption into the ballot box, they are obviously bringing their “perceptions” and not “complete and true” information about corruption. TI implies as much when they say: “The interplay of corruption and inequality also feeds populism. When traditional politicians fail to tackle corruption, people grow cynical. Increasingly, people are turning to populist leaders who promise to break the cycle of corruption and privilege. Yet this is likely to exacerbate – rather than resolve – the tensions that fed the populist surge in the first place.”
Interestingly, Supreme Court First Amendment law has actually picked up on this thread. Political speech, including political spending, has First Amendment protections. However, the Court has long recognized that Congress can write campaign finance laws which tackle both corruption and the “appearance” of it. As the Court said in 1970 in Buckely v. Valeo: Congress can legitimately conclude that “the avoidance of the appearance of improper influence is also critical . . . if confidence in the system of representative Government is not to be eroded to a disastrous extent.” While what can create the appearance of corruption has been construed narrowly in cases like Citizens United, Congress can still (at least on paper) combat perceptions.
All this is to say, I agree that the knock on the CPI as not observing corruption reality is misplaced. Reality matters, but in a democracy so do our perceptions of it.
Dear C Millian, It is very critical to say in analysis of CPI reporting from TI, when this organization is marking a worldwide ‘perception’ of corruption and the anti-corruption measures that may reflect on comparative study of PCI. But following welcoming writings and critics revealed data are not as per ‘aspiration’ from scientific point of view of anti-corruption research.
Scientific research and the reality never meet the target point, due to the humanoid modification or ‘biasness’.
US is the leading country of this planet in mainly economic and power application. Anti-corruption research also been polished by US -led economy as well. Research chronology on corruption is the major example as a whole. Origin of corruption and its’ scientific method of eradication, or at least regulation is the mode of avoidance in scientific publications. No clear mandate from the recent luxurious 17IACC conference, at Panama- is the vivid prove of anti-corruption standard, when Panama state is accused (documented) of world black-money store house.
From the FORBES research, world is informed that major accumulation of world wealth is in USA, when this very country is projecting worldwide terrorism-related semi-world wars.
In such a grave environment, TI is bold enough to place US in 18th place from 16th of 2015.
The democracy is been provided some rules and regulation through the “Constitution”, which is fragile enough in the hands of politicians,- present US-president is practicing it in daily basis. Then amendments in constitution is nothing but a play-match to protect criminals of world-economy.
The mass poverty ridden world is striving with the least ‘straw’ in the ocean.