Tag Archives: measurement

Assessing Corruption: Do We Need a Number?

Posted on by
9

As GAB readers are aware, I’ve occasionally used this platform to complain about widely-repeated corruption statistics that appear to be, at best, unreliable guesstimates misrepresented as precise calculations—and at worst, completely bogus. (The “$1 trillion in annual bribe payments” figure would be an example of the former; the “corruption costs the global economy $2.6 trillion per year” is an example of the latter.) I recognize that, in the grand scheme of things, made-up statistics and false precision are not that big a deal. After all, the anticorruption community faces 1,634 problems that are more important than false precision, and in any event 43% of all statistics quoted in public debates are completely made up. Yet my strong instincts are that we in the anticorruption community ought to purge these misleading figures from our discussions, and try to pursue not only the academic study of corruption, but also our anticorruption advocacy efforts, using a more rigorous and careful approach to evidence.

But perhaps I’m wrong about that, or at least naïve. A few months ago, after participating in a conference panel where some of the other speakers invoked the “corruption costs $2.6 trillion” figure, I was having a post-panel chat with another one of the panelists (an extremely smart guy who runs the anticorruption programs at a major international NGO), and I was criticizing (snarkily) the tendency to throw out these big but not-well-substantiated numbers. Why, I asked, can’t we just say, “Corruption is a really big problem that imposes significant costs?” We’ve got plenty of research on that point, and—a few iconoclastic critics aside—the idea that corruption is a big problem seems to have gained widespread, mainstream acceptance. Who really cares if the aggregate dollar value of annual bribe payments is $1 trillion, $450 billion, $2.3 trillion, or whatever? Why not just say, corruption is bad, here’s a quick summary of the evidence that it does lots of damage, and move on? My companion nodded, smiled, and said something along the lines of, “Yeah, I see what you’re saying. But as an advocate, you need to have a number.”

We didn’t get to continue our conversation, but that casual remark has stuck with me. After all, as I noted above, this person is extremely smart, insightful, and reflective, and he has lots of experience working on anticorruption advocacy at a very high level (a kind of experience that I, as an Ivory Tower academic, do not have). “As an advocate, you need to have a number.” Is that right? Is there a plausible case for continuing to open op-eds, speeches, policy briefs, and so forth with statements like, “Experts estimate that over $1 trillion bribes are paid each year, costing the global economy over $2.6 trillion,” even if we know that those numbers are at best wildly inaccurate? (This question, by the way, is closely related to an issue I raised in a post last year, that arose out of a debate I had with another advocate about the legal interpretation of the UN Convention Against Corruption.)

I thought I’d use this post as an opportunity to raise that question with our readers, in the hopes of both getting some feedback (especially from our readers with first-hand experience in the advocacy and policymaking communities) and provoking some conversations on this question, even if people don’t end up writing in with their views. And to be clear, I’m not just interested in the narrow question of whether we should keep using the $2.6 billion or $1 trillion estimates. I’m more generally curious about the role (and relative importance) of seemingly precise “big numbers” in anticorruption advocacy work. Do we really need them? Why? And is what we gain worth the costs?

It’s Time to Abandon the “$2.6 Trillion/5% of Global GDP” Corruption-Cost Estimate

Posted on by
19

In my post a couple weeks back, I expressed some puzzlement about the source of the widely-quoted estimate that corruption costs the global economy approximately $2.6 trillion, or roughly 5% of global GDP. I was hoping that someone out there in GAB Reader-Land would be able to point me to the source for this figure (as several GAB readers helpfully did when I expressed similar puzzlement last year about the source for the related estimate that there are approximately $1 trillion in annual bribe payments). Alas, although several people made some very insightful comments (some of which are in the public comment thread with the original post), this time it seems that nobody out there has been able to point me to a definitive source.

I’ve done a bit more poking around (with the help of GAB readers and contributors), and here’s my best guess as to where the $2.6 trillion/5% of GDP number comes from: Continue reading →

The Amount of Bribery and the Cost of Bribery Are Not the Same

Posted on by
1

I’ve posted before (see here, here, and here) about some of my concerns regarding the accuracy of the estimates people sometimes throw around about the total amount of bribes paid each year (sometimes given in absolute terms, sometimes as a percentage of global GDP, but in all cases based on dubious extrapolations from suspect data). But for the moment I want to put those concerns aside to make another point: Even if we knew the total amount of bribes paid, that would not necessarily tell us much of anything about how much bribery costs society. (And that’s true even if we limited attention to economic costs, narrowly construed.) This is not an original point – lots of people have made it, and indeed it’s fairly obvious when you stop to think about it. Yet I keep seeing references to estimates of the amount of bribery that treat these figures as if they were measures of the cost of bribery. (For examples, see here, here, here, here, and here.) But that’s just not right. Continue reading →

At Last, A Good News Corruption Story

Posted on by
3

It seems that not a day goes by without some gloomy story about corruption appearing in the popular media or online. “Corruption on the rise in Africa poll as governments seen failing to stop it” says a new TI study.  “In Mexico, 200 million acts of corruption a year” the Mexican Competitiveness Institute reports.  Monday’s Washington Post editorial proclaims that “Mali’s corruption hindered its efforts to fight terror,” and the subtitle of a best-selling book warns that it is not only Malians who are at risk but that corruption “Threatens Global Security as Well.”

With all this bad news it was a surprise to discover a recent good news story about corruption.  The news is doubly surprising as it comes out of three unexpected places: Ghana Kenya, and Uganda.  Even better, rather than broad generalizations drawn from a handful of selected anecdotes, the good news in Professors Rebecca Dizon-Ross, Pascaline Dupas, and Jonathan Robinson’s July 2015 “Governance and the Effectiveness of Public Health Subsidies” paper rests on a careful, clever empirical study that employs rigorous scientific methods.  The only bad news about the paper is that it is on a remote internet site beyond the ken of most web browsers.  For readers whose browsers don’t travel to the National Bureau of Economic Research’s web site, a potted summary follows. Continue reading →

A Quick (Partial) Fix for the CPI

Posted on by
2

A regular readers of this blog know, I’ve been quite critical of the idea that one can measure changes in corruption (or even the perception of corruption) using within-country year-to-year variation in the Transparency International Corruption Perceptions Index (CPI). To be clear, I’m not one of those people who like to trash the CPI across the board – I actually think it can be quite useful. But given the way the index is calculated, there are big problems with looking at an individual country’s CPI score this year, comparing it to previous years, and drawing conclusions as to whether (perceived) corruption is getting worse or better. Among the many problems with making these sort of year-to-year comparisons is the fact the sources used to calculate any individual country’s CPI score may change from year to year, and the fact that a big, idiosyncratic movement in an individual source can have an outsized influence on the change in the composite score. (For more discussion of these points, see here, here, and here.) Also, while TI does provide 90% confidence intervals for its yearly estimates, the fact that confidence intervals overlap does not necessarily mean that there’s no statistically significant difference between the scores (an important point I’ll confess to sometimes neglecting in my own prior discussions of these issues).

Although there are lots of other problems with the CPI, and in particular with making over-time CPI comparisons, I think there’s a fairly simple procedure that TI (or anybody working with the TI data) could implement to address the problems just discussed. Since TI will be releasing the 2015 CPI within the next month, I thought this might be a good time to lay out what I think one ought to do to evaluate whether there have been statistically significant within-country changes in the CPI from one year to another. (I should say up front that I’m not an expert in statistical analysis, so it’s entirely possible I’ve screwed this up in some way. But I think I’ve got the important parts basically right.) Here goes: Continue reading →

Guest Post: Why We Should Be Excited About SDG 16

Posted on by
7

GAB is delighted to welcome back Daniel Dudis, Senior Policy Director at Transparency International-USA, who contributes the following guest post:

On September 25th, the United Nations adopted the Sustainable Development Goals (SDGs). The SDGs identify development priorities and set measurable targets for progress that are to be met by 2030. They also replace the Millennium Development Goals (MDGs), adopted in 2000 and set to expire at the end of this year. The MDGs were aimed primarily at improving living conditions in developing countries, and focused on reducing extreme poverty and improving health, education, sanitation, and nutrition. Unfortunately, progress towards achieving the MDGs has been uneven at best. Notably absent from the MDGs were any commitments on improving governance or reducing corruption. Given that in most countries, government is the primary service provider for healthcare, education, and sanitation, and that government provides nutrition assistance and sets economic policy, the absence of any commitments to improve governance or reduce corruption was a notable blind spot. Honest, accountable, efficient government is the foundation upon which economic development and improved service delivery are built.

Happily, goal 16 of the SDGs fills this lacuna. Goal 16, which seeks to promote just, peaceful, and inclusive societies, includes (among other governance-related targets) significant reductions in illicit financial flows, progress on the recovery and return of stolen assets, and substantial reductions in corruption and bribery.

It is easy to be skeptical about the utility of ambitious international agreements such as the SDGs. Indeed, Matthew’s post last week, which criticized the Goal 16’s anticorruption targets on the grounds that they are ill-suited to quantitative measurement of progress, and Rick’s post yesterday, exemplify that view. Such skepticism, however, is misplaced. The inclusion of these targets in Goal 16 of the SDGs is an important step forward as it represents a clear endorsement by the community of nations that good governance and the fight against corruption are integral parts of the global development agenda. Continue reading →

Am I the Only One Who’s Not So Excited About SDG 16?

Posted on by
12

This Friday, over 190 world leaders are scheduled to gather at the UN headquarters in New York City for the UN Sustainable Development Summit to endorse a new set of “Sustainable Development Goals” (SDGs) to be achieved over the next 15 years. The SDGs are a follow-up to the Millennium Development Goals (MDGs), but the SDGs are much more expansive and cover a wider range of topics. Most relevant to the anticorruption community is Goal 16 (“Promote Peaceful and Inclusive Societies for Sustainable Development, Provide Access to Justice for All and Build Effective, Accountable and Inclusive Institutions at all Levels”), and in particular SDG “Target” 16.5 (“substantially reduce corruption and bribery in all their forms”).

There seems to be a lot of excitement among anticorruption activists and reformers about Goal 16 and Target 16.5 (see here, here, and here)—but to be honest, I’m not sure why. Indeed, I tend to think that the formal endorsement of anticorruption as part of the SDGs will do little good, and the inclusion of Target 16.5 might, if anything, be counterproductive. Continue reading →

Are We Underestimating the Extent of Bribery in the World?

Posted on by
8

The astounding figure Richard Rose and Caryn Peiffer report in their new book, Paying Bribes for Public Services, that almost one quarter of world’s population or 1.6 billion people, recently paid a bribe would suggest the answer to the question above is a resounding “No.”  The 1.6 billion figure sounds so fantastically large that the suspicion arises that it is one of those gauzy numbers conjured up using shaky assumptions and questionable sources to capture headlines rather than advance learning.  Yet recent research by the World Bank’ Art Kraay and University of Maryland Professor Peter Murrell shows that, if anything, the Rose and Peiffer 1.6 billion number is low.

Their figure is based on the most solid of evidence: interviews by phone or in-person where respondents are asked whether they had to pay a bribe to obtain a public service.  Transparency International’s 2013 Global Corruption Barometer, a main source for the 1.6 billion number, is an example.  Surveyors first ask respondents if they or anyone else in their household has had any contact in the past 12 months with anyone associated with any of eight government services: i) the education system, ii) the judiciary, iii) medical or health services, iv) the police, v) registry and permit services, vi) utilities, vii) tax collection or, viii) land service.  If the answer is yes, the surveyor then asks:

In your contact or contacts have you or anyone living in your household paid a bribe in any form in the past 12 months?    

What could be a more reliable way to gather evidence of bribery?  Instead of asking what people think about bribery or what their perceptions of bribery or corruption are, they are asked about their own personal experience, or that of close relatives, with the crime of bribery.  The rub comes with the last phrase in the preceding sentence: the respondent is being questioned about “the crime of bribery.”

Continue reading →

The 2014 CPI Data Demonstrates Why, Even Post-2012, CPI Scores Cannot Be Compared Over Time

Posted on by
5

A little while back, I expressed some skepticism about whether Transparency International’s Corruption Perceptions Index (CPI) scores can be compared across time, even after TI changed its methodology in 2012 and claimed that its new scores would now be comparable across years.  More recently, I criticized TI’s 2014 CPI for burying the information on the margins of error associated with the CPI values, and for wrongly asserting that changes in the CPI score between 2013 and 2014 for certain countries (most notably China) were substantively meaningful.  (In fact, not only does the change in China’s score between 2013 and 2014 seem not to be statistically significant, but the change was due almost entirely to the dropping of a source in which China did abnormally well in 2013, and an abnormally large movement in a single other source.) I decided to follow up on this by taking a closer look at the other ten countries that TI singled out as having experienced significant CPI changes (in either direction) between 2013 and 2014.

Upon closer examination, I’m even more certain that CPI scores cannot be compared over time. I’m also more confident in my judgment that TI has been unforgivably sloppy — and downright misleading — in how it, and its representatives, have portrayed the substantive significance of these CPI changes. It turns out that the problem I found with the China calculations was not unusual. For almost all of the eleven countries TI identified as big movers, the CPI changes were driven by (1) the addition or elimination of sources from year to year for particular countries, and/or (2) abnormally large (indeed, implausibly large) movements in a single source. Until TI fixes its methodology, the safest thing to do is to ignore year-to-year changes in the CPI. And for the sake of preserving its own integrity and credibility, TI should either (A) persuasively explain why I am wrong in my analysis of the data (in which case I will gladly concede error), or (B) issue some sort of retraction or correction to its earlier press releases, and either drop the claim that post-2012 CPI scores can be compared across time or fix its methodology going forward.

Allow me to elaborate my analysis of the data: Continue reading →

Combating Corruption in Uganda or Merely Displacing it: The World Bank’s Public Expenditure Tracking Survey

Posted on by
5

A World Bank-initiated effort to reduce corruption in school funding in Uganda is widely, and rightly, celebrated for its results (click here and here for background).  In the early nineteen nineties on average 87 percent of the monies the Ugandan central government budgeted for textbooks and other school supplies “leaked out” somewhere between departing the Finance Ministry and arriving at the school house front door.  Yearly data revealed that 73 percent of the schools received less than five percent of the monies to which they were entitled, and only ten percent received more than half.  The 1996 Bank project had an immediate effect on the rate of losses.  By 1999 the government found schools were receiving on average 95 percent of what they were supposed to receive, and a 2002 World Bank study likewise showed a sharp drop in fund leakage.

The dramatic improvement is attributed to the enormous publicity the data on losses garnered.  Parents were outraged and the government and donor agencies embarrassed.  Within the development community, the Uganda Public Expenditure Tracking Survey, as the work to dig out and publicize the loss data became known, has been enormously influential, the story becoming a parable for how to fight corruption.  A Uganda-like PETS project is now routinely prescribed for attacking corruption in public expenditures, and a Google search on “Uganda PETS” yields over 100,000 hits and returns some 20,000 citations on Google scholar.

But for all the attention the effort has generated, there is evidence that it may not have had any impact on the level of corruption in Uganda.  It is possible that all it did was force those raking funds off the school fund program to turn elsewhere.  The Uganda PETS thus may simply have displaced the corruption in the school funding program rather than ending it. Continue reading →