A few weeks back, I posted a skeptical commentary about the integration of anticorruption into the new Sustainable Development Goals and associated targets, in particular Target 16.5 (“substantially reduce corruption and bribery in all their forms”). Rick was even harsher. The premise of most of my criticism (and Rick’s) was that progress on Target 16.5 was likely to be measured using changes in countries’ scores on Transparency International’s Corruption Perceptions Index (CPI). It turns that this premise was (probably) incorrect.

I had based my assumption on the lengthy report released last June by the Sustainable Development Solutions Network (SDSN)—a report which had been commissioned by the UN’s Inter-Agency Expert Group on SDG Indicators (IAEG-SDG). But as Transparency International Senior Policy Coordinator Craig Fagan helpfully pointed out in his comment on Rick’s post, the more recent official information released by IAEG-SDG in September 2015 does not indicate that the CPI will be used as the principal measure for Target 16.5. Rather, the IAEG-SDG document lists as the proposed indicator the “percentage of persons who had at least one contact with a public official, who paid a bribe to a public official, or were asked for a bribe by these public officials, during the last 12 months.” (The relevant material is on page 225.) This still isn’t finalized, but it certainly appears that the IAEG is poised to endorse an experience/survey-based measure for Target 16.5, rather than the CPI-style perception index.

Is this perfect? No, certainly not. But it’s a lot better than what I’d feared. A few further thoughts on this:

• First, some might object to this sort of experience-based measure as too narrow. And indeed it’s true that there’s a bit of a mismatch between the broadly-stated target (reducing corruption and bribery in all their forms) and the proposed measure, which at best only measures certain forms of petty bribery. But this doesn’t bother me that much. It’s usually better, in my view, to adopt a narrower target that one can actually measure than to try to capture a whole host of disparate phenomena in a single quantitative index. And even though petty bribery is not the only, or even the most damaging, form of corruption, it’s certainly important. Perhaps the title of the target should be adjusted (though that may not be possible at this stage). But the fact that we might ultimately be unable to measure reductions in corruption and bribery in all their forms doesn’t seem like a good argument against measuring reductions in corruption and bribery in some of their forms.
• Second, a more serious concern—which Rick pointed out in his response to Craig’s comment—is that there are a number of quite important methodological concerns about these experience surveys. The most significant of these is that people do not always answer honestly when asked about illicit activities like corruption. That by itself might not be so serious, except that the tendency to be candid about such matters may vary substantially across countries/cultures, and even within countries may be correlated with characteristics of the respondents in ways that bias the results. There’s quite a bit of research now that confirms that this is a problem (see here, here, here, here, and here). Rick points out (rightly, in my view) that this makes it extremely difficult to do cross-country comparisons using experience surveys: If 20% of respondents in Country X say they paid a bribe within the last year, and only 10% of respondents in Country Y say that they did so, this could mean that bribery is more widespread in Country X, but it could also mean that respondents in Country X are more willing to admit paying bribes, even if the actual bribery rates are the same (or even if bribery is actually more prevalent in Country X). But I’m less convinced than Rick that this is a fatal problem for using experience surveys to measure within-country progress toward Target 16.5. If the other factors that might affect willingness to answer truthfully remain constant over time within individual countries, then changes in reported bribery experience over time within a single country may still be meaningful, even if comparisons across countries are not meaningful. In this regard, experience surveys and the CPI can perhaps be thought of as opposites in terms of what they can or can’t be used for: for all its flaws, I think the CPI can still be used to get a (rough) sense of cross-country variation in (perceived) corruption levels, but within-country changes in the CPI are not meaningful; with corruption experience surveys, I tend to think that within-country changes can provide useful information, but cross-country comparisons do not.
• Third, though, the preceding optimistic assessment of the utility of over-time, within-country changes in bribery experience survey results is based on the premise that there are no other within-country changes are affecting the willingness to respond truthfully to bribery questions. But that’s not necessarily the case. For example, suppose a government launches an aggressive anticorruption campaign, including a significant educational component. This could cause self-reported bribery to decrease, even if actual bribery does not change, if the campaign increases the shame associated with corruption. Alternatively, such a campaign could also increase self-reported bribery (again, even if actual bribery is not changing), if, for instance, the campaign causes people to realize that certain payments that they thought were official charges are actually unlawful bribes, or if the campaign makes people feel empowered to complain openly about bribe demands. To my knowledge, as yet we have very little research examining the factors that may affect willingness to answer corruption experience questions, so we would need to be very cautious when interpreting results until we know more.
• Fourth, and related to the above points, it should probably go without saying that in some countries it will be very difficult to conduct such surveys, or to have any confidence in the results.

Still, although there are a number of serious issues that still need to be addressed, I tend to think it would be a good thing if we got more data, on a yearly basis from a large number of countries, about individual/household/firm-level experience with bribery. As long as we interpret that data cautiously, it might indeed provide useful information. And if the adoption of SDG 16 and Target 16.5 provides an impetus to invest more in gathering such data (and to scrutinize it more carefully, and to pay more attention to concerns like some of those sketched above), then that would be all to the good. So while I still find myself unable to muster the enthusiasm for SDG 16 that I observe among many in the anticorruption community (I still don’t get why people are referring to this as a transformative, watershed, breakthrough moment), I’m much more optimistic that Target 16.5 won’t do harm, and might actually do a modest amount of good. I’m grateful to Craig for correcting my original error, and I look forward to seeing what the IAEG-SDG actually ends up doing.