Transparency International’s Anti-Corruption Pledge Tracker Is Badly Flawed. It Needs To Be Redone from Scratch.

In May 2016, at the London Anticorruption Summit sponsored by then-Prime Minister David Cameron, participating countries issued declarations announcing a variety of commitments—some new, some continuations of existing policies—to further the fight against international corruption. Of course, all too often governments fail to follow through on their grandiose promises, so I was heartened by Transparency International’s announcement, in September 2016, that it had gone through all the country declarations, compiled a spreadsheet identifying each country’s specific promises, and would be monitoring how well each country was following through on its commitments.

Last month, a year after TI published the spreadsheet documenting the list of summit commitments, TI released a report and an interactive website that purport to track whether countries have followed through on those commitments. So what do we learn from this tracking exercise?

Alas, the answer is “almost nothing.” TI’s “Anti-Corruption Pledge Tracker,” in its current form, is a catastrophic failure—a slapdash, amateurish collection of arbitrary, often inconsistent judgments, unsupported by anything that resembles serious research, and (ironically) non-transparent. This is all the more surprising—and disappointing—given the fact that TI has done so much better in producing similar assessment tools in other contexts. Indeed, at least one such recent tool—TI’s Government Defense Anti-Corruption Index—provides a model for what the Pledge Tracker could and should have looked like. Given the importance of tracking countries’ fulfillment of their summit pledges, and TI’s natural position as a leader on that effort, I dearly hope that TI will scrap the Pledge Tracker in its current form, go back to the drawing board, and do a new version.

I know that sounds harsh, and perhaps it seems excessive. But let me explain why I don’t find the Pledge Tracker, in its current form, worthy of credence. Continue reading

London Anticorruption Summit–Country Commitment Scorecard, Part 1

Well, between the ICIJ release of the searchable Panama Papers/Offshore Leaks database, the impeachment of President Rousseff in Brazil, and the London Anticorruption Summit, last week was quite a busy week in the world of anticorruption. There’s far too much to write about, and I’ve barely had time to process it all, but let me try to start off by focusing a bit more on the London Summit. I know a lot of our readers have been following it closely (and many participated), but quickly: The Summit was an initiative by David Cameron’s government, which brought together leaders and senior government representatives from over 40 countries to discuss how to move forward in the fight against global corruption. Some had very high hopes for the Summit, others dismissed it as a feel-good political symbolism, and others were somewhere in between.

Prime Minister Cameron stirred things up a bit right before the Summit started by referring to two of the countries in attendance – Afghanistan and Nigeria – as “fantastically corrupt,” but the kerfuffle surrounding that alleged gaffe has already received more than its fair share of media attention, so I won’t say more about it here, except that it calls to mind the American political commentator Michael Kinsley’s old chestnut about how the definition of a “gaffe” is when a politician accidentally tells the truth.) I’m going to instead focus on the main documents coming out of the Summit: The joint Communique issued by the Summit participants, and the individual country statements. There’s already been a lot of early reaction to the Communique—some fairly upbeat, some quite critical (see, for example, here, here, here, and here). A lot of the Communique employs fairly general language, and a lot of it focuses on things like strengthening enforcement of existing laws, improving international cooperation and information exchange, supporting existing institutions and conventions, and exploring the creation of new mechanisms. All that is fine, and some of it might actually turn out to be consequential, but to my mind the most interesting parts of the Communique are those that explicitly announce that intention of the participating governments to take pro-transparency measures in four specific areas:

  1. Gathering more information on the true beneficial owners of companies (and possibly other legal entities, like trusts), perhaps through a central public registry—which might be available only to law enforcement, or which might be made available to the general public (see Communique paragraph 4).
  2. Increasing transparency in public contracting, including making public procurement open by default, and providing usable and timely open data on public contracting activities (see Communique paragraph 9). (There’s actually a bit of an ambiguity here. When the Communique calls for public procurement to be “open by default,” it could be referring to greater transparency, or it could be calling for the use of open bidding processes to increase competition. Given the surrounding context, it appears that the former meaning was intended. The thrust of the recommendation seems to be increasing procurement transparency rather than increasing procurement competition.)
  3. Increasing budget transparency through the strengthening of genuinely independent supreme audit institutions, and the publication of these institutions’ findings (see Communique paragraph 10).
  4. Strengthening protections for whistleblowers and doing more to ensure that credible whistleblower reports prompt follow-up action from law enforcement (see Communique paragraph 13).

Again, that’s far from all that’s included in the Communique. But these four action areas struck me as (a) consequential, and (b) among the parts of the Communique that called for relatively concrete new substantive action at the domestic level. So, I thought it might be a useful (if somewhat tedious) exercise to go through each of the 41 country statements to see what each of the Summit participants had to say in each of these four areas. This is certainly not a complete “report card,” despite the title of this post, but perhaps it might be a helpful start for others out there who are interested in doing an assessment of the extent of actual country commitments on some of the main action items laid out in the Communique. So, here goes: a country-by-country, topic-by-topic, quick-and-dirty summary of what the Summit participants declared or promised with respect to each of these issues. (Because this is so long, I’m going to break the post into two parts. Today I’ll give the info for Afghanistan–Malta, and Thursday’s post will give the info for Mexico–United States). Continue reading

The 2014 CPI Data Demonstrates Why, Even Post-2012, CPI Scores Cannot Be Compared Over Time

A little while back, I expressed some skepticism about whether Transparency International’s Corruption Perceptions Index (CPI) scores can be compared across time, even after TI changed its methodology in 2012 and claimed that its new scores would now be comparable across years.  More recently, I criticized TI’s 2014 CPI for burying the information on the margins of error associated with the CPI values, and for wrongly asserting that changes in the CPI score between 2013 and 2014 for certain countries (most notably China) were substantively meaningful.  (In fact, not only does the change in China’s score between 2013 and 2014 seem not to be statistically significant, but the change was due almost entirely to the dropping of a source in which China did abnormally well in 2013, and an abnormally large movement in a single other source.) I decided to follow up on this by taking a closer look at the other ten countries that TI singled out as having experienced significant CPI changes (in either direction) between 2013 and 2014.

Upon closer examination, I’m even more certain that CPI scores cannot be compared over time. I’m also more confident in my judgment that TI has been unforgivably sloppy — and downright misleading — in how it, and its representatives, have portrayed the substantive significance of these CPI changes. It turns out that the problem I found with the China calculations was not unusual. For almost all of the eleven countries TI identified as big movers, the CPI changes were driven by (1) the addition or elimination of sources from year to year for particular countries, and/or (2) abnormally large (indeed, implausibly large) movements in a single source. Until TI fixes its methodology, the safest thing to do is to ignore year-to-year changes in the CPI. And for the sake of preserving its own integrity and credibility, TI should either (A) persuasively explain why I am wrong in my analysis of the data (in which case I will gladly concede error), or (B) issue some sort of retraction or correction to its earlier press releases, and either drop the claim that post-2012 CPI scores can be compared across time or fix its methodology going forward.

Allow me to elaborate my analysis of the data: Continue reading