This Friday, over 190 world leaders are scheduled to gather at the UN headquarters in New York City for the UN Sustainable Development Summit to endorse a new set of “Sustainable Development Goals” (SDGs) to be achieved over the next 15 years. The SDGs are a follow-up to the Millennium Development Goals (MDGs), but the SDGs are much more expansive and cover a wider range of topics. Most relevant to the anticorruption community is Goal 16 (“Promote Peaceful and Inclusive Societies for Sustainable Development, Provide Access to Justice for All and Build Effective, Accountable and Inclusive Institutions at all Levels”), and in particular SDG “Target” 16.5 (“substantially reduce corruption and bribery in all their forms”).
There seems to be a lot of excitement among anticorruption activists and reformers about Goal 16 and Target 16.5 (see here, here, and here)—but to be honest, I’m not sure why. Indeed, I tend to think that the formal endorsement of anticorruption as part of the SDGs will do little good, and the inclusion of Target 16.5 might, if anything, be counterproductive.
To understand my skepticism, it’s perhaps helpful to begin by emphasizing that what makes the SDGs (like their MDG predecessors) distinctive is that they purport not only to announce in general terms what the international community views as important, but to establish specific (measurable) goals and targets, with a 15-year timeframe for hitting those targets, using quantitative indicators. (That’s the only reason I can think of why SDG Target 16.5 would offer anything that the UN Convention Against Corruption, and its associated peer review mechanism, doesn’t already provide.) The question, then, is whether that approach is the right approach to take to an objective like reducing corruption.
So how will the UN and other monitors be able to tell whether countries have succeeded in “substantially reduc[ing]” all forms of corruption and bribery? Although the UN has not yet officially announced how progress will be measured, we do have last June’s official report to the Secretary General from the Sustainable Development Solutions Network (SDSN), which had the task of preparing a report for the UN’s Inter-Agency and Expert Group on SDG Indicators (IAEG-SDG). The SDSN’s 225 page report, the product of a year and half of work and consultations with thousands of experts, therefore may be our best source of insight into how the UN will monitor progress toward achieving Target 16.5. And the indicator that the SDSN report proposes to measure progress toward this target is…
…<drum roll please>…
The Transparency International Corruption Perceptions Index (CPI).
Seriously. I swear I’m not making this up. Apparently, the SDSN Brain Trust is advising the UN to endorse, as part of the SDG evaluation progress, improvement in each country’s CPI score—presumably by some amount, relative to the 2014 baseline CPI score. This is despite the fact that, as a general matter, the SDSN argues against the use of composite indexes because “they require more complex data collection methods, and often rely on imputation for missing variables and arbitrary weighting,” and also emphasizes that indicators should “be consistent to enable measurement over time” (which the CPI is not). And indeed, the UNODC’s own National Anticorruption Strategy Guide (which, full disclosure, I helped draft, and which was published after the SDSN Report) specifically says that changes in a country’s CPI score should not be used to measure progress toward achieving anticorruption goals, for a variety of very sound reasons.
To be fair, the SDSN acknowledges that corruption is difficult to measure, that “objective data tends to be highly incomplete and difficult to compare,” and that the CPI indicator is problematic. (The SDSN Report gives this indicator a “C” grade, noting that it “will be very difficult or infeasible within the time frame” to gather all the necessary data.) But those concessions naturally invite the question whether it’s at all appropriate to use over-time changes in the CPI score to measure progress on addressing corruption. (The answer, as I’ve argued several times on this blog, is no, no, no, and no.) And of course, even if inter-temporal CPI comparisons were legit, in order for the SDG target to be meaningful, we’d still need to be able to come up with appropriate target improvement levels for each country, and it’s not at all clear how to do that. Moreover, as many, many people (and TI itself) have pointed out, the CPI measures perceptions, which may not correspond to reality—or may be slow to react to genuine changes—and have very large statistical margins of error, making these scores doubtful candidates for SDG-style progress assessments.
Of course, the SDSN Report is just a proposal (albeit quite an extensive and likely influential one), and it remains to be seen what indicators the IAEG-SDG will ultimately adopt. Is there anything better than the CPI for this sort of over-time anticorruption progress assessment? Well, there’s the Worldwide Governance Indicators control-of-corruption index, and the International Country Risk Guide published by Political Risk Services, both of which may be slightly more suitable for over-time comparisons, though only slightly. But really the larger question here, and the source of my inability to muster a lot of enthusiasm for SDG 16, is whether anticorruption is really all that well-suited to the sort of quantitative target approach associated with the MDGs (and presumably extended in the SDGs). This is not, I hasten to add, due to any objection to quantitative assessment in the anticorruption area. I’m all for that – indeed, we should have much more of it. The question is whether the particular approach reflected in the SDGs (including Target 16.5), requiring a national-level quantitative metric used to assess progress over time at the national level is a useful and appropriate means of evaluating success (and ensuring accountability) in the context of anticorruption efforts. My tentative answer to that question is no. I’m happy to be convinced otherwise, but I want to hear an argument about how integrating CPI improvements (or something similar) into the SDGs is likely to be helpful, as opposed to distracting, confusing, and potentially misleading.
GAB | The Global Anticorruption Blog 
It seems to me that the best argument for the Corruption Perception Index is ease of use. The “data” is simply presented and easily digestible, which perhaps makes a complex problem feel more manageable and accessible, and might also thus attract more engagement from the general public. Transparency International’s name recognition and good public reputation may also seem like good reasons for association.
But I do not think these are good justifications. Better data is better data. If the Worldwide Governance Indicators and the International Country Risk Guide both offer better information, that ought to be the definitive selection criterion. Given the shortcomings of year-to-year analysis in corruption indices, as explained in UNODC’s own National Anticorruption Strategy Guide, it is puzzling that the Sustainable Development Goals could not better incorporate insights from other UN bodies.
I share your skepticism “whether anticorruption is really all that well-suited to the sort of quantitative target approach associated with the MGDs.” Yet it seems an institution like the UN needs some kind of measurable data to assess its efforts. I am curious to know more about the indicators you would endorse to measure the success (or failure) of anticorruption efforts. The UNODC Strategy Guide suggests investigations, audits, and randomized, controlled spot checks to test policies in a particular office or community. Could these be translated into useful indicators for an undertaking like the SDGs (accepting the SGDs are a worthwhile undertaking in the first place)? Quantitative analysis may certainly have its limits, but doesn’t the UN need some kind of measurement system? Is this a situation to accept “good enough” quantitative analysis? Target 16.5 of “substantially reducing all forms of corruption and bribery” is so incredibly lofty it becomes almost meaningless. With such a vague goal, nearly any data set could assist. That could be cause for disillusionment, but it also means that non-quantitative indicators should be perfectly welcome.
Ease of use was also the first reason that came to my mind, perhaps with the somewhat related benefit of familiarity. Countries trying to set anticorruption goals for themselves seem to turn to the CPI with remarkable frequency. My intuition is that the same would go for individuals and groups who care about the SDGs’ goals but are not involved directly in this sort of field—more people seem to know about the CPI than any other qualitative measure of corruption. What’s more, the average observer, if I had to guess, probably interprets the CPI country rankings as aligning in broad strokes with their ideas of which countries “rank” where. That passing familiarity might make some observers question why the CPI wouldn’t be used. Still, these are not good reasons to choose the CPI if there is a better alternative.
This could be a shot, but could legitimizing the CPI by increasingly marking it as the standard corruption-measuring tool encourage TI to improve its methodology? (I just checked their funding website, which mentions receiving support from “multilateral institutions,” among others, though I’m not sure if that refers to international organizations.) Assuming that there are legitimate needs for qualitative research on corruption, and assuming that the CPI is the or one of the better starting grounds, I wonder what could be done to improve measures that would be more accurate for comparison.
As the to the SDGs, my sense is that most of them are not particularly suited to quantitative measurement. If the evaluative methods marked for likely use in Target 16.5 are deficient, I’d be willing to guess that other measures will be too. I agree that Target 16.5, by itself, may not do much that isn’t being done already. At the same time, what the SDGs may have going for them is an interconnected approach to tackling issues that often go hand-in-hand. Better education, food security, and jobs access could increase a country’s ability to tackle entrenched corruption. Yes, the lofty, numerous goals are likely to fall short. At the same time, integrated planning, as encouraged by the UN, and aims for smaller gains across the board could offer something a bit different.
Lots of interesting thoughts here, but I’ll limit myself to two quick reactions to your last two paragraphs.
First, while it would be nice if the greater legitimacy and prominence attached to year-to-year CPI comparisons got TI to take the critiques of the data (as used for that purpose) more seriously, I fear that it would actually have either no effect, or the opposite effect: legitimizing yearly CPI changes as the “gold standard” for measuring corruption reduction, making it less politically palatable to emphasize the problems with doing that.
Second, I think your sense of the purpose of the SDGs may be right, though if so, they’re actually much further from the original MDG concept than I’d appreciated. But if you read the whole SDSN report, and a lot of the other material on the SDGs, it seems to me the desire to come up with measurable, quantitative benchmarks and targets is still very much there. But that instinct–we want to measure progress rigorously and quantitatively, and could both individual countries and the international community accountable using those objective indicators–is in deep tension with another laudable instinct: to adopt a more capacious understanding of (sustainable) development, that includes what have traditionally been seen as “soft” factors (like governance), but that are almost certainly both intrinsically valuable and important for achievement of other goals. The problem is that it’s hard to achieve both of these objectives at once, and my beef with the SDGs (or at least some of them, including Target 16.5) is that they try to do both and may end up doing neither well.
I do share some of your skepticism and there is much more to be said about the political and methodological aspects of the SDG debate and in particular the Goal 16. Thus, you are not he only one being skeptical. As I was involved in the arts of the SDG Goal 16 process I must say that it is more important politically. Let me remind that MDGs had nothing to say about governance, crime and corruption. Now the SDGs do contain one Goal that brings up these issues on an international agenda and links them with much broader developmental issues. This is important and should receive the support. The issue of specific targets and indicators is much more complicated. Targets the same as the goals are the results of a political compromise and are not methodologically sound to enable good indicators and measurement. On the other hand the isssue of the indicators is primariliy in the hands of the UN Statistical Office, the UN Statistica Commission and the national statistical offices – this is indeed a big problem. But that is the reults of the prevailing ideological belief that only the quantitative data are the evidence to be taken into account. Yet, this is not so much the problem of the SDGs as it is the problem of the prevailing ideological paradigm in social sciences today. This paradigm became “corrupt by onedimensional approach” – that is the quantitative and usually a very limited and bad approach.
Ugi ZVekic
Very fair points, and I get the political significance of including governance. (Though I’m still slightly puzzled about WHY, exactly, it’s so important. It’s not like governance wasn’t already on the international development agenda. And corruption has certainly emerged as a high-profile issue on the international agenda more generally. Do we really think the politics of international development going forward will be that much different simply because Goal 16 is included, if we put aside for the moment and aspiration to measure it rigorously and quantitatively?)
I guess my reaction to your quite reasonable comments is similar to my reaction to Kaitlin’s comment above: It seems that the current SDGs are the result of a political compromise between two conflicting instincts: the impulse to use quantitative benchmarks, and the impulse to include factors that are inherently very difficult to quantify (at least at the national level). And like many political compromises, the final result strikes me as somewhat incoherent, which is perhaps what I was really trying to get at in my post.
Much good ground has been covered in the comments above, so I’ll only add in a sidenote (one that Kait kind of alludes to): given that the SDGs seem to have taken the approach of including just about everything anyone could possibly want, I’m much less convinced about the value of including anticorruption efforts among them. With a finite number of resources, if you prioritize everything, you’re really prioritizing nothing, so I’m not sure that there will be be that much political value (I suppose the counterargument is that if nearly everything EXCEPT corruption would included, that would be problematic, but that just means we’ve avoided sliding back in our anti-corruption political discourse, not that a real step forward has been taken).
In either scenario you mention, it sounds like neither inclusion nor exclusion may lead to a real step forward. Then, the question is, as I think is alluded to in the post, whether inclusion actually marks a step backward. Skepticism of the wisdom of quantifying anti-corruption efforts has been repeated in several comments, as has skepticism of the specific CPI mechanism. But if the act of being measured against some index spurs action or provides a measure against which to work, then in this case, some quantifiable measure may be preferable to none at all, especially if including nothing on corruption would be noticeable and problematic. It’s possible that lending further credibility to an imperfect mechanism that may be hailed as a gold standard could in the long term have negative effects on priorities and perceptions, and more so if other measurements don’t effectively counsel against complacency. On the other hand, if the adopted index acts more as a “starting point” to track development, it might serve some positive purpose, if not as well as another measure might.
Pingback: Development Channel » This Week in Markets and Democracy: U.S. Fights Corruption, Preventing Mass Atrocities, and More
Pingback: Does our corruption look big in this? SDG 16 and the problem of measurement - Devpolicy Blog from the Development Policy Centre
Pingback: Connecting Health and the Fight Against Corruption | Anti Corruption Digest
Pingback: Innovative or Ineffective?: Performance-Based Lending as an Anticorruption Tool | GCNI
Pingback: The U4 Proxy Challenge and the Search for New Corruption Indicators | Anti Corruption Digest