Tag Archives: measurement

At Last, A Good News Corruption Story

Posted on by
3

It seems that not a day goes by without some gloomy story about corruption appearing in the popular media or online. “Corruption on the rise in Africa poll as governments seen failing to stop it” says a new TI study.  “In Mexico, 200 million acts of corruption a year” the Mexican Competitiveness Institute reports.  Monday’s Washington Post editorial proclaims that “Mali’s corruption hindered its efforts to fight terror,” and the subtitle of a best-selling book warns that it is not only Malians who are at risk but that corruption “Threatens Global Security as Well.”

With all this bad news it was a surprise to discover a recent good news story about corruption.  The news is doubly surprising as it comes out of three unexpected places: Ghana Kenya, and Uganda.  Even better, rather than broad generalizations drawn from a handful of selected anecdotes, the good news in Professors Rebecca Dizon-Ross, Pascaline Dupas, and Jonathan Robinson’s July 2015 “Governance and the Effectiveness of Public Health Subsidies” paper rests on a careful, clever empirical study that employs rigorous scientific methods.  The only bad news about the paper is that it is on a remote internet site beyond the ken of most web browsers.  For readers whose browsers don’t travel to the National Bureau of Economic Research’s web site, a potted summary follows. Continue reading →

A Quick (Partial) Fix for the CPI

Posted on by
2

A regular readers of this blog know, I’ve been quite critical of the idea that one can measure changes in corruption (or even the perception of corruption) using within-country year-to-year variation in the Transparency International Corruption Perceptions Index (CPI). To be clear, I’m not one of those people who like to trash the CPI across the board – I actually think it can be quite useful. But given the way the index is calculated, there are big problems with looking at an individual country’s CPI score this year, comparing it to previous years, and drawing conclusions as to whether (perceived) corruption is getting worse or better. Among the many problems with making these sort of year-to-year comparisons is the fact the sources used to calculate any individual country’s CPI score may change from year to year, and the fact that a big, idiosyncratic movement in an individual source can have an outsized influence on the change in the composite score. (For more discussion of these points, see here, here, and here.) Also, while TI does provide 90% confidence intervals for its yearly estimates, the fact that confidence intervals overlap does not necessarily mean that there’s no statistically significant difference between the scores (an important point I’ll confess to sometimes neglecting in my own prior discussions of these issues).

Although there are lots of other problems with the CPI, and in particular with making over-time CPI comparisons, I think there’s a fairly simple procedure that TI (or anybody working with the TI data) could implement to address the problems just discussed. Since TI will be releasing the 2015 CPI within the next month, I thought this might be a good time to lay out what I think one ought to do to evaluate whether there have been statistically significant within-country changes in the CPI from one year to another. (I should say up front that I’m not an expert in statistical analysis, so it’s entirely possible I’ve screwed this up in some way. But I think I’ve got the important parts basically right.) Here goes: Continue reading →

Guest Post: Why We Should Be Excited About SDG 16

Posted on by
7

GAB is delighted to welcome back Daniel Dudis, Senior Policy Director at Transparency International-USA, who contributes the following guest post:

On September 25th, the United Nations adopted the Sustainable Development Goals (SDGs). The SDGs identify development priorities and set measurable targets for progress that are to be met by 2030. They also replace the Millennium Development Goals (MDGs), adopted in 2000 and set to expire at the end of this year. The MDGs were aimed primarily at improving living conditions in developing countries, and focused on reducing extreme poverty and improving health, education, sanitation, and nutrition. Unfortunately, progress towards achieving the MDGs has been uneven at best. Notably absent from the MDGs were any commitments on improving governance or reducing corruption. Given that in most countries, government is the primary service provider for healthcare, education, and sanitation, and that government provides nutrition assistance and sets economic policy, the absence of any commitments to improve governance or reduce corruption was a notable blind spot. Honest, accountable, efficient government is the foundation upon which economic development and improved service delivery are built.

Happily, goal 16 of the SDGs fills this lacuna. Goal 16, which seeks to promote just, peaceful, and inclusive societies, includes (among other governance-related targets) significant reductions in illicit financial flows, progress on the recovery and return of stolen assets, and substantial reductions in corruption and bribery.

It is easy to be skeptical about the utility of ambitious international agreements such as the SDGs. Indeed, Matthew’s post last week, which criticized the Goal 16’s anticorruption targets on the grounds that they are ill-suited to quantitative measurement of progress, and Rick’s post yesterday, exemplify that view. Such skepticism, however, is misplaced. The inclusion of these targets in Goal 16 of the SDGs is an important step forward as it represents a clear endorsement by the community of nations that good governance and the fight against corruption are integral parts of the global development agenda. Continue reading →

Am I the Only One Who’s Not So Excited About SDG 16?

Posted on by
12

This Friday, over 190 world leaders are scheduled to gather at the UN headquarters in New York City for the UN Sustainable Development Summit to endorse a new set of “Sustainable Development Goals” (SDGs) to be achieved over the next 15 years. The SDGs are a follow-up to the Millennium Development Goals (MDGs), but the SDGs are much more expansive and cover a wider range of topics. Most relevant to the anticorruption community is Goal 16 (“Promote Peaceful and Inclusive Societies for Sustainable Development, Provide Access to Justice for All and Build Effective, Accountable and Inclusive Institutions at all Levels”), and in particular SDG “Target” 16.5 (“substantially reduce corruption and bribery in all their forms”).

There seems to be a lot of excitement among anticorruption activists and reformers about Goal 16 and Target 16.5 (see here, here, and here)—but to be honest, I’m not sure why. Indeed, I tend to think that the formal endorsement of anticorruption as part of the SDGs will do little good, and the inclusion of Target 16.5 might, if anything, be counterproductive. Continue reading →

Are We Underestimating the Extent of Bribery in the World?

Posted on by
8

The astounding figure Richard Rose and Caryn Peiffer report in their new book, Paying Bribes for Public Services, that almost one quarter of world’s population or 1.6 billion people, recently paid a bribe would suggest the answer to the question above is a resounding “No.”  The 1.6 billion figure sounds so fantastically large that the suspicion arises that it is one of those gauzy numbers conjured up using shaky assumptions and questionable sources to capture headlines rather than advance learning.  Yet recent research by the World Bank’ Art Kraay and University of Maryland Professor Peter Murrell shows that, if anything, the Rose and Peiffer 1.6 billion number is low.

Their figure is based on the most solid of evidence: interviews by phone or in-person where respondents are asked whether they had to pay a bribe to obtain a public service.  Transparency International’s 2013 Global Corruption Barometer, a main source for the 1.6 billion number, is an example.  Surveyors first ask respondents if they or anyone else in their household has had any contact in the past 12 months with anyone associated with any of eight government services: i) the education system, ii) the judiciary, iii) medical or health services, iv) the police, v) registry and permit services, vi) utilities, vii) tax collection or, viii) land service.  If the answer is yes, the surveyor then asks:

In your contact or contacts have you or anyone living in your household paid a bribe in any form in the past 12 months?    

What could be a more reliable way to gather evidence of bribery?  Instead of asking what people think about bribery or what their perceptions of bribery or corruption are, they are asked about their own personal experience, or that of close relatives, with the crime of bribery.  The rub comes with the last phrase in the preceding sentence: the respondent is being questioned about “the crime of bribery.”

Continue reading →

The 2014 CPI Data Demonstrates Why, Even Post-2012, CPI Scores Cannot Be Compared Over Time

Posted on by
5

A little while back, I expressed some skepticism about whether Transparency International’s Corruption Perceptions Index (CPI) scores can be compared across time, even after TI changed its methodology in 2012 and claimed that its new scores would now be comparable across years.  More recently, I criticized TI’s 2014 CPI for burying the information on the margins of error associated with the CPI values, and for wrongly asserting that changes in the CPI score between 2013 and 2014 for certain countries (most notably China) were substantively meaningful.  (In fact, not only does the change in China’s score between 2013 and 2014 seem not to be statistically significant, but the change was due almost entirely to the dropping of a source in which China did abnormally well in 2013, and an abnormally large movement in a single other source.) I decided to follow up on this by taking a closer look at the other ten countries that TI singled out as having experienced significant CPI changes (in either direction) between 2013 and 2014.

Upon closer examination, I’m even more certain that CPI scores cannot be compared over time. I’m also more confident in my judgment that TI has been unforgivably sloppy — and downright misleading — in how it, and its representatives, have portrayed the substantive significance of these CPI changes. It turns out that the problem I found with the China calculations was not unusual. For almost all of the eleven countries TI identified as big movers, the CPI changes were driven by (1) the addition or elimination of sources from year to year for particular countries, and/or (2) abnormally large (indeed, implausibly large) movements in a single source. Until TI fixes its methodology, the safest thing to do is to ignore year-to-year changes in the CPI. And for the sake of preserving its own integrity and credibility, TI should either (A) persuasively explain why I am wrong in my analysis of the data (in which case I will gladly concede error), or (B) issue some sort of retraction or correction to its earlier press releases, and either drop the claim that post-2012 CPI scores can be compared across time or fix its methodology going forward.

Allow me to elaborate my analysis of the data: Continue reading →

Combating Corruption in Uganda or Merely Displacing it: The World Bank’s Public Expenditure Tracking Survey

Posted on by
5

A World Bank-initiated effort to reduce corruption in school funding in Uganda is widely, and rightly, celebrated for its results (click here and here for background).  In the early nineteen nineties on average 87 percent of the monies the Ugandan central government budgeted for textbooks and other school supplies “leaked out” somewhere between departing the Finance Ministry and arriving at the school house front door.  Yearly data revealed that 73 percent of the schools received less than five percent of the monies to which they were entitled, and only ten percent received more than half.  The 1996 Bank project had an immediate effect on the rate of losses.  By 1999 the government found schools were receiving on average 95 percent of what they were supposed to receive, and a 2002 World Bank study likewise showed a sharp drop in fund leakage.

The dramatic improvement is attributed to the enormous publicity the data on losses garnered.  Parents were outraged and the government and donor agencies embarrassed.  Within the development community, the Uganda Public Expenditure Tracking Survey, as the work to dig out and publicize the loss data became known, has been enormously influential, the story becoming a parable for how to fight corruption.  A Uganda-like PETS project is now routinely prescribed for attacking corruption in public expenditures, and a Google search on “Uganda PETS” yields over 100,000 hits and returns some 20,000 citations on Google scholar.

But for all the attention the effort has generated, there is evidence that it may not have had any impact on the level of corruption in Uganda.  It is possible that all it did was force those raking funds off the school fund program to turn elsewhere.  The Uganda PETS thus may simply have displaced the corruption in the school funding program rather than ending it. Continue reading →

More on the “News from Nowhere” Problem in Anticorruption Research

Posted on by
8

One of my all-time favorite academic papers — which should be required reading not only for those who work on anticorruption, but on any topic where people casually throw around statistics — is Marc Galanter‘s 1993 article News from Nowhere: The Debased Debate on Civil Justice. Professor Galanter’s paper doesn’t have anything directly to do with international corruption. Rather, he sets out to debunk a series of widely-held but mostly-false beliefs about civil litigation in the United States, and in the process he traces the origins of many of the statistics often cited in debates about that topic. He finds that many of these statistics come from, well, nowhere. Here’s my favorite example: Around the time Professor Galanter was writing, it was common to hear claims that the civil justice system costs $80 billion in direct litigation costs; indeed, that figure appeared in an official report from the President’s Council on Competitiveness. The report’s only source for that estimate, however, was an article in Forbes; Forbes, in turn, had drawn the figure from a 1988 book by Peter Huber. But Huber himself hadn’t done any direct research on the costs of the system. Rather, Huber’s only source for the $80 billion figure was an article in Chief Executive magazine, which reported that at a roundtable discussion, a CEO claimed that “it’s estimated” (he didn’t say by whom) that insurance liability costs industry $80 billion per year. So: A CEO throws out a number at a roundtable discussion, without a source, it gets quoted in a non-scholarly magazine, repeated (and thus “laundered”) in what appears to be a serious book, and then picked up in the popular press and official government reports as an important and troubling truth about the out-of-control costs of the US civil justice system.

I thought about Galanter’s book the other day when I was reading the Poznan Declaration on “Whole-of-University Promotion of Social Capital, Health, and Development.” The Declaration itself is about getting universities to commit to integrating anticorruption and ethics into their programs; I may have something to say about the substance of the declaration itself in a later post. But the following assertion in the Declaration caught my eye: “Despite the relative widespread implementations of anti-corruption reforms and institutional solutions, no more than 21 countries have enjoyed a significant decrease in corruption levels since 1996, while at the same time 27 countries have become worse off.” Wow, I thought, that seems awfully precise, and if it’s true it’s very troubling. Despite the fact that I spend a fair amount of time reading about the comparative study of corruption, that statistic is news to me. It turns out, though, that it’s news from nowhere. Continue reading →

National Anticorruption Strategies: Lessons from the Asia-Pacific Region

Posted on by
4

The 173 nation that have ratified the U.N. Convention Against Corruption are obliged by article 5 to “develop and implement or maintain effective, coordinated anticorruption policies . . . .”  Many meet this requirement by adopting a national anticorruption strategy, and such strategies are so common that the World Bank, the U4 Anticorruption Resource Centre, and Transparency International have all published works advising how to develop and implement them.  The latest, and most useful entry, comes from the Bangkok office of the U.N. Development Program.   Released December 9,  Anticorruption Strategies: Understanding What Works, What Doesn’t and Why — Lessons from the Asia-Pacific Region draws on the experience of 14 countries in the region  to  help guide policymakers wanting to promulgate a national strategy or revise an existing one. Continue reading →

Transparency International Makes Its Data Less Transparent: Why TI Should Be Ashamed of Its 2014 CPI Report

Posted on by
10

For all its flaws, I’ve long been of the view that Transparency International’s annual Corruption Perceptions Index (CPI) has, on balance, made a positive contribution to our understanding of corruption, and the fight against it. (A couple of my sympathetic treatments can be found here and here.) Although some in the media (and, depressingly, some in academic and policy circles) misuse the index, TI has generally been quite clear about what the CPI numbers do and do not tell us.  And to its great credit, TI has proven remarkably receptive to criticism: each year TI’s annual CPI report has become better, clearer, more nuanced, and more transparent in its limitations.

Until this year. The 2014 CPI came out yesterday, and I’m disappointed at how TI has taken a big step backward, making the meaning of its scores less transparent, and choosing to play for catchy headlines rather than to deepen understanding. Continue reading →