Matthew wrote last month about the February competition the U4 Anticorruption Resource Center and the United Kingdom’s Department for International Affairs sponsored to spur creation of new measures of corruption. What he did not say is that one subtext for the contest was the growing frustration with the use of Transparency International’s Corruption Perceptions Index (CPI) to measure the actual level of corruption in a country.
This is not the fault of TI. The organization is careful to say on its web site that its index does not measure the actual extent of corruption and goes to great lengths to explain how the index is constructed, stressing that it is a ranking of how corrupt countries are perceived to be using the opinions of business people and country experts. Indeed the title selected, “Corruption Perceptions Index,” couldn’t be any clearer about what is being measured. But journalists and academics frequently treat the index as if it measured actual corruption, rather than perceived corruption, or assume that perceptions match reality fairly closely. And that’s where problems may arise.
When the CPI first appeared in 1995, the distinction between perceptions and actual levels of corruption seemed of little concern: a close connection between the two was assumed. In other words, a measure of corruption perceptions was taken to be a good proxy, or stand-in, for a measure of actual corruption.
Recent research, however, has shown this assumption does not hold. In a 2006 paper Daniel Treisman reported that in countries where perceived corruption levels were high, there was little relationship between perceptions of corruption and citizens’ experience with bribery. He also found that perceptions of corruption levels in a country were influenced by the country’s level of economic development and democratization; with all other things equal, wealthier, more democratic countries are perceived to be less corrupt than poorer, non-democratic ones. Work by Dilyan Donchev and Gergely Ujhelyi reaches the same conclusion about perception bias, adding that survey respondents’ education, age, and employment also affect their perceptions of corruption. Other studies showing the divergence between actual and perceived levels of corruption can be found here, here, and here.
The response of many anticorruption activists to these findings is often: “So what? Hasn’t the index been extraordinarily successful in prompting action on corruption?” The answer is an unqualified “yes.” A country’s low score on the index draws media attention and often serves as a rallying point for reformers. The index’ annual appearance puts tremendous pressure on policymakers in low scoring countries to act, and the world community owes Transparency International a debt of gratitude for creating and publicizing the index.
But it is one thing to serve as a stimulus to action, and quite another to serve as a measure of the effectiveness of that action. As much as the CPI has succeeded at the former, it is ill-suited for the latter. And as the priority has shifted from pressuring countries to act to ensuring the actions they take are effective, attention to measurement has become ever more critical. In a later post I will provide some examples of how continued use of the index as a performance measure can frustrate progress on the very objective for which it was created.