As discussed on this blog and elsewhere, one of the big concerns about the most popular cross-country datasets on corruption (the Transparency International Corruption Perceptions Index (CPI), the World Bank Institute’s Worldwide Governance Indicators (WGI), etc.) is that they are based (largely or entirely) on perceptions of corruption. As Rick noted in a recent post, and as the critical literature has pointed out ad nauseam, perceptions, while perhaps important in their own right, are not necessarily based in reality. Indeed, some recent research (including, but certainly not limited to, nice papers by Claudio Weber Abramo, by Mireille Razafindrakoto and Francois Rouband, and by Richard Rose and William Mishler) indicates that national corruption perceptions are only weakly correlated with survey results asking about individuals’ personal experience with bribery. This raises serious questions about whether the perception-based indicators are useful either for general assessment or for testing hypotheses about the causes or consequences of corruption.
But might there be more objective measures that could be used to assess whether the corruption perceptions indices are picking up something real? Off the top of my head, I can think of four quite clever recent papers that demonstrate a strong correlation between a subjective corruption perception index and some more objective measure of dishonest behavior. I’m sure there are more, but let me note the four examples that I can think of, and then say a bit on what this might mean for the use of perception-based indicators in empirical corruption research.
- First, a well-known paper Raymond Fisman and Edward Miguel examined the propensity of UN diplomats to abuse their diplomatic immunity to violate New York City parking rules, and found that this propensity was greater for diplomats from countries ranked as more corrupt according to the WGI Control of Corruption index. (However, it’s worth noting that once New York got permission from the State Department to pull the plates of serial violators, compliance by all diplomats increased.)
- Second, another paper by Fisman and Shang-Jin Wei examined evidence of illegal export of antiques and other cultural property, by comparing the reported value of exports with the reported value of imports. (The logic is that there’s a strong incentive to misrepresent a relic’s provenance and value at the export stage, in order to avoid national export restrictions, but the incentive is to be truthful at the import stage.) The gap between the value of exports and imports (controlling for a variety of factors) is an implicit measure of unlawful activity, and probably corruption. The paper finds that the size of the gap is strongly (and negatively) correlated with the WBI Control of Corruption index, even after controlling for a variety of other factors.
- Third, a forthcoming paper by Jason DeBacker, Bradley Heim, and Ahn Tran looks at corporate tax evasion by foreign firms operating in the United States. The paper finds that, at least for small- and medium-sized firms, the extent of tax evasion is negatively correlated with the CPI score for the firm’s home country (again, after controlling for a range of other factors, including the IRS’s own assessment of how likely an audit is to uncover additional tax liability).
- Fourth, a recent working paper by Laarni Escresa and Lucio Picci (which I posted about earlier this week) looks at enforcement data for the FCPA and similar foreign anti-bribery laws, and finds that the likelihood of a particular country’s officials being implicated in an FCPA action (after controlling for U.S. exports to that country, and several other factors) is strongly correlated with the CPI. (My earlier post raises some questions about the assumptions underlying this paper, but I will put those aside for the moment.)
So, it looks like there’s at least some evidence that perception-based corruption measures are fairly strongly correlated with more objective quantifications of a range of dishonest behaviors (none of which look like they have much to do with one another, except insofar as all of them involve some form of cheating). What should we take away from this? Here are a few tentative thoughts:
- First, maybe the strong form of the perceptions-are-not-reality criticism is overblown. True, perceptions are not reality, but we now have some decent (though admittedly imperfect) evidence that perceptions are at least correlated with reality. Now, this doesn’t change the fact that several studies (noted above) found little correlation between corruption perceptions and self-reported victimization. But there’s some evidence that people don’t answer such surveys honestly (even when asked about victimization rather than willing participation in bribery)—see, for example, Aart Kraay and Peter Murrell’s recent paper. And other research, for example a recent paper by Nicholas Charron, finds that at least in some countries, self-reported corruption victimization really does correlate strongly with corruption perceptions.
- Second, the fact that perception-based indices seem to be correlated with more objective corruption measures does NOT mean that those perception measures are unbiased. This is an important distinction—perhaps one that’s obvious to the statistics nerds, but not necessarily to others who may be interested in using research that relies on perception measures. Suppose, for example, that we want to see whether democracy is correlated with control of corruption. Suppose further (1) that the CPI is correlated with true corruption, (2) that democracy is NOT correlated with true corruption, but (3) that for other reasons (such as bias on the part of the evaluators) democracies tend to get better CPI scores. A simple analysis would report a positive correlation between democracy and corruption, even though (by supposition in this example) that correlation is spurious. So proponents of the perception measures might want to hold off on doing their victory dance, even if recent research does establish that the perception-based measures seem to be picking up something “real.”
- Finally, it’s worth considering two other common criticisms of the perception-based measures: (1) that they create an illusion of precision (though this is less true of the WGI, which include margins of error, than for the CPI), and (2) that they aren’t based on a clearly-articulated, theoretically-informed definition of “corruption” (as Michel noted in a previous post). It seems to me that the validity of these criticisms is not affected one way or the other by research showing that perception-based corruption indices are correlated with various objective measures of different sorts of dishonesty.