Tag Archives: measurement

Guest Post: Corruption Among Development NGOs, Part 1–Getting the Facts

Posted on by
6

Roger Henke, Chairman of the Board of the Southeast Asia Development Program (SADP), a development grantmaker based in Cambodia, contributes the following guest post (the first in a three-part series):

Compared with media attention to corruption among public officials and corporate interests, corruption in the non-profit sector is virtually ignored (though a recent GAB post on NGO corruption in India is a notable exception). This lack of interest is matched by the absence of sustained substantive debates about the sources of NGO sector corruption and the effectiveness of remedial interventions. My own experience with these issues derives from my involvement with the NGO sector in Cambodia. Corruption within our own house is a regular topic of informal conversation, and also makes it into our periodic sectoral assessments (though often through oblique references to concerns like “weak financial systems” and the “lack of checks and balances”). However, there are no efforts at all to go beyond these anecdotes and self-reported “weaknesses” to gather systematic, externally validated evidence about levels of corruption, let alone about issues like costs of corruption or the way it correlates with characteristics of the NGO sector that would offer entry points for positive change.

Given the comparative importance of development aid channeled through the NGO sector in countries like Cambodia, this lack of attention to NGO corruption is unfortunate. Admittedly, gathering information on local NGO (LNGO) corruption is challenging. Yet there are potentially useful sources of information that have not been exploited. For example, LNGOs are funded by grantmakers, and these grantmakers (often criticized by LNGOs for their cumbersome administrative requirements and time-consuming monitoring visits) are a possible source of data about LNGO fraud and its correlates. Additionally, the audit firms with an LNGO client base are another possible source of information.

In 2014, to test the willingness of grantmakers and audit firms to share information on their LNGO partners and NGO client base, we at SADP piloted a grantmaker and audit firm survey. The results were promising enough to repeat and expand the exercise in 2015. In this second grantmaker survey, 18 out of 26 grantmakers approached agreed to participate, and 13 of those 18 shared LNGO partner-level information (for a sample of 93 LNGOs). The grantmaker survey queried incidence and seriousness of (1) financial management problems, (2) governance problems, and (3) fraud. (In order to maximize participation, the survey prioritized brevity and simplicity over depth of information.) The audit firm survey (in which four of the five firms approached agreed to participate) asked only for some aggregate data (total number of LNGO audited, number of audits that identified fraud, number of audits that flagged serious financial system issues, etc.). Admittedly, neither the sample of grantmakers nor the sample of LNGOs is statistically representative of Cambodia’s NGO sector, but the surveys provide more valid information about corruption in development NGOs in Cambodia than has previously been available. And the quantitative picture emerging from the combination of these two data-sources about the organizational quality of Cambodian LNGOs is both revealing and disheartening. Interested readers should check out the full report; the most important findings are as follows: Continue reading →

The Case for Including Sextortion Measures in TI’s CPI

Posted on by
1

In a recent post, I called for the creation of an international index of sexual corruption. While I believe that such an index will have an effect standing alone, I also believe that such an index, once created, should be included as one of the sources used to construct composite indexes such as Transparency International’s Corruption Perceptions Index (CPI). As most GAB readers are likely aware, the CPI is does not reflect TI’s own independent assessment of corruption perception, but rather aggregates corruption perception measures from a range of other sources. These other sources, however, all measure perceptions of monetary corruption, such as bribery and embezzlement. But, as TI itself acknowledges, sexual corruption may not correlate well with other forms of corruption, meaning that an index like the CPI may give us an incomplete and misleading picture.

The exclusion of sexual corruption is not TI’s fault; there are currently no global comparative measures of perceptions of sexual corruption for TI to incorporate. Indeed, this gap is precisely why I advocate the creation of an international sexual corruption perceptions index. Of course, even if such an index is created, it would be a separate question whether the results ought to be included in the CPI. I believe it should be.

Continue reading →

WAGs: What’s the Harm?

Posted on by
6

GAB is pleased to publish this Guest Post by Maya Forstater, well-known analyst on business and sustainable development, on a topic of continuing concern to scholars and activists working on corruption and development matters.

Are unreliable guesstimates  and made-up statistics mildly irritating, indispensably powerful  or potentially dangerous in the public debates on corruption? The topic comes up so often on the Global Anti-Corruption Blog that it has been given its own own three-letter acronym: WAGs (or Wild Ass Guesses).

Those at the sharp end of advocacy maintain, with some justification, that in the battle for attention, an arrestingly big number makes all the difference. But as Rick has argued, overinflated figures can also cause harm.

Something similar happens on the related topic of tax and illicit flows. One example of this is the widespread belief that ‘developing countries lose three times more to the tax avoidance by multinational companies than they receive in aid’. This much quoted WAG gives the impression of huge potential gains for the poorest countries, but is based on a chain of misunderstandings .  In practice the magnitudes of revenues at stake are likely to be several times smaller than aid  for the countries where that comparison matters.

Similarly, broad estimates of illicit flows or the scale of the black economy (“trillions”) are often presented in ways that suggest that the sums to be gained from tackling corporate tax avoidance are larger than any serious analysis supports.

I have written about these big numbers previously in a paper published by the Centre for Global Development here (or here  for the short version).

But what harm do such numbers do, compared to their power at getting people talking about the issues? Is it really worth pointing out misunderstandings and myths in pursuit of a more rigorous and careful approach to evidence? (Or as I have been asked‘ Do you ever wonder how much you help the tax abusers?’)

I see four key dangers from inflated perceptions of the numbers:  Continue reading →

Measurement Brings Action: The Need for a Global Sexual Corruption Index

Posted on by
6

Sexual corruption is a scourge, to varying degrees, in almost every country–from immigration officials demanding sex for green cards, to U.N. soldiers using their power to force themselves on refugees or the local population they are supposed to be protecting, to police officers who demand sex in exchange for not arresting someone. The International Association of Women Judges has been trying to bring attention to this “sextortion” problem, with some limited success: Transparency International (TI) describes sextortion as a form of corruption, and last September’s International Anti-Corruption Conference devoted a high-profile session to discussing this issue.

Yet despite this increasing recognition that this sort of sexual corruption is indeed corruption–the abuse of public power for private gain–the major international indexes used to measure corruption, such as TI’s corruption perception index (CPI) (and the underlying studies used to generate the CPI), focus overwhelmingly on material corruption–principally monetary bribery and embezzlement–not the abuse of public power to extort sexual favors from victims. This is a problem: As we have seen over and over again (both in the corruption context, and in other contexts such as the Millennium Development Goals (MDGs)), for better or worse, national-level country ratings drive action. Right now, a country that wishes to improve its global standing on corruption currently has little incentive to tackle sexual corruption. And there is no separate, easy-to-understand metric that calls attention to how well (or poorly) countries are doing, relative to one another, in addressing that problem.

It is time for that to change. It is time to create a Global Sexual Corruption Index. Continue reading →

The 2015 CPI and Year-to-Year Changes: A Definite Improvement, But Problems Remain

Posted on by
1

As most people who follow this blog are likely aware, Transparency International released the 2015 Corruption Perceptions Index (CPI) last week. There is, of course, a lot to talk about here, and I’m sure many commentators and scholars will spend a lot of time poring over the new data and debating its significance. Given my previous criticisms of the CPI’s suggestion that scores for the same country can be compared across time (see here, here, here, and here), that was naturally the first thing I focused on. I was hoping that TI might take up some version of my suggestion to report statistical confidence intervals in an easy-to-see place in the main data table, or, even better, test for statistically significant changes in scores across years. Alas, TI didn’t do either of those things. (The confidence intervals are still available, but you have to download the data to find them.) TI did, however, report that since 2012, some countries had improved, while others had deteriorated. In particular, TI noted three countries (Greece, Senegal, and the UK) had improved their CPI scores since 2012, while five countries (Australia, Brazil, Libya, Spain, and Turkey) had seen a notable worsening.

Because of last year’s fiasco with China (where TI emphasized a decline in China’s CPI score that turned out to be bogus), I was initially skeptical. So, I went ahead and implemented the procedure that I outlined in my post from a few months ago to see whether, for these eight countries, there really was a genuine, statistically meaningful change in the CPI score. I was pleasantly surprised to discover that in all eight of the countries that TI identified, the change in the CPI score between 2012 and 2015 was indeed statistically significant at conventional levels, and do not seem to have been driven by the addition or subtraction of sources in the later year, or by a large anomalous jump in a single source. (Though it’s perhaps worth noting that in the case of Brazil – which TI particularly emphasizes in its press release – the change is just barely significant at conventional levels, and of the seven sources used to construct the score, although four indicate moderate to large declines, two show no change and one actually rates Brazil as improving slightly from 2012 to 2015.) So, while I still have a number of criticisms (about which more below), I’ll gladly give credit where credit is due: In this year’s publicity materials, TI has indeed identified countries where there is statistically significant change in CPI scores, generally driven by changes in several of the underlying data sources. I hope that in future years, TI will go further (and save me some time) by simply including in the main data table not only the confidence interval for the current year, but also a simple three-category indicator (up, down, null) for whether there has been a statistically significant change in the CPI in the past three years. (This is important because of the way the CPI is covered by mainstream journalists: Though researchers might dig into the data tables, most journalists or casual readers just look for year-to-year changes.)

Now, I did say I still had some concerns, so in the interest of continued constructive engagement, let me lay out why I still don’t think we should treat within-country year-to-year changes in CPI scores as terribly meaningful: Continue reading →

Guest Post: A New Additional Indicator for Measuring Progress Toward SDG 16

Posted on by
3

GAB is delighted to welcome back Dieter Zinnbauer, Programme Manager at Transparency International, who contributes the following guest post:

A very interesting discussion has evolved on this blog (see here, here, here, and here), and in the wider world (for example, see here), on about the indicators that should be used to measure progress toward the Sustainable Development Goals (SDGs) goals for improving governance and reducing corruption (Goal 16). There are already some very good suggestions on the table, including the use of Transparency International’s Global Corruption Barometer (GCB) to measure progress toward Target 16.5, on reducing corruption and bribery in all their forms. (TI has used the GCB since 2005 to compile one of the largest data troves on the detailed experience with corruption of households and individuals around the world. Using a GCB-type indicator for the bribery dimension of SDG 16.5 is supported by a wide variety of stakeholders, including the World Bank, UNDP, and Save the Children.)

Yet most of the indicators proposed so far, including the GCB, speak to very specific aspects of corruption (such as bribery) and don’t quite do justice to Goal 16’s broad ambitions and its emphasis on public accountability. So to spice up this stew a bit, let me suggest another possible indicator, one that complement to some of the ideas that are already on the table. My proposed indicator of progress toward SDG 16 is as follows:

What percentage of national-level parliamentarians (and perhaps top level members of the executive) have made assets, income, and interest disclosures (AIIDs) in a format that is publicly accessible online at sufficient level of detail, in timely manner, and in a machine-readable data format.

Using AIID as an additional SDG 16 indicator might at first seem to be a step backwards, since such an indicator measures “outputs” rather than “outcomes.” But let me try to convince you that in fact AIID would be an extremely useful complementary indicator for progress toward SDG 16: Continue reading →

More Phony Numbers–This Time on the Anticorruption Impact of Open Data

Posted on by
5

OK, I know I’m beating a dead horse. Within the last month I’ve already posted several times (see here, here, and here) about bogus anticorruption statistics, as has Rick. And I promise that after this post, I’ll move on to other topics. But I can’t help commenting on this latest release from Transparency International, criticizing the recent World Economic Forum (WEF) meeting for not explicitly addressing corruption. As its lead example, TI faults the WEF for not addressing issues like open data (and openness more generally). I’m sympathetic to TI’s policy position, but in making the case, TI asserts, “One study suggests that open data could reduce the costs of corruption by about 10 percent.”

I was curious (and, admittedly, skeptical) about yet another seemingly precise estimate of something that’s inherently hard to measure. So I clicked on the link to the “one study” that “suggests” that open data technologies would reduce the costs of corruption by 10%. This “study” is actually a report (really, an advocacy document) from an Australian consulting firm (Lateral Economics), commissioned by a philanthropic fund (the Omidyar Network) that invests in open data initiatives. How does this “study” reach its conclusion that open data could reduce the costs of corruption by 10%? I will now quote in full the entirety of the evidence and analysis supporting that conclusion: Continue reading →

Assessing Corruption: Do We Need a Number?

Posted on by
9

As GAB readers are aware, I’ve occasionally used this platform to complain about widely-repeated corruption statistics that appear to be, at best, unreliable guesstimates misrepresented as precise calculations—and at worst, completely bogus. (The “$1 trillion in annual bribe payments” figure would be an example of the former; the “corruption costs the global economy $2.6 trillion per year” is an example of the latter.) I recognize that, in the grand scheme of things, made-up statistics and false precision are not that big a deal. After all, the anticorruption community faces 1,634 problems that are more important than false precision, and in any event 43% of all statistics quoted in public debates are completely made up. Yet my strong instincts are that we in the anticorruption community ought to purge these misleading figures from our discussions, and try to pursue not only the academic study of corruption, but also our anticorruption advocacy efforts, using a more rigorous and careful approach to evidence.

But perhaps I’m wrong about that, or at least naïve. A few months ago, after participating in a conference panel where some of the other speakers invoked the “corruption costs $2.6 trillion” figure, I was having a post-panel chat with another one of the panelists (an extremely smart guy who runs the anticorruption programs at a major international NGO), and I was criticizing (snarkily) the tendency to throw out these big but not-well-substantiated numbers. Why, I asked, can’t we just say, “Corruption is a really big problem that imposes significant costs?” We’ve got plenty of research on that point, and—a few iconoclastic critics aside—the idea that corruption is a big problem seems to have gained widespread, mainstream acceptance. Who really cares if the aggregate dollar value of annual bribe payments is $1 trillion, $450 billion, $2.3 trillion, or whatever? Why not just say, corruption is bad, here’s a quick summary of the evidence that it does lots of damage, and move on? My companion nodded, smiled, and said something along the lines of, “Yeah, I see what you’re saying. But as an advocate, you need to have a number.”

We didn’t get to continue our conversation, but that casual remark has stuck with me. After all, as I noted above, this person is extremely smart, insightful, and reflective, and he has lots of experience working on anticorruption advocacy at a very high level (a kind of experience that I, as an Ivory Tower academic, do not have). “As an advocate, you need to have a number.” Is that right? Is there a plausible case for continuing to open op-eds, speeches, policy briefs, and so forth with statements like, “Experts estimate that over $1 trillion bribes are paid each year, costing the global economy over $2.6 trillion,” even if we know that those numbers are at best wildly inaccurate? (This question, by the way, is closely related to an issue I raised in a post last year, that arose out of a debate I had with another advocate about the legal interpretation of the UN Convention Against Corruption.)

I thought I’d use this post as an opportunity to raise that question with our readers, in the hopes of both getting some feedback (especially from our readers with first-hand experience in the advocacy and policymaking communities) and provoking some conversations on this question, even if people don’t end up writing in with their views. And to be clear, I’m not just interested in the narrow question of whether we should keep using the $2.6 billion or $1 trillion estimates. I’m more generally curious about the role (and relative importance) of seemingly precise “big numbers” in anticorruption advocacy work. Do we really need them? Why? And is what we gain worth the costs?

It’s Time to Abandon the “$2.6 Trillion/5% of Global GDP” Corruption-Cost Estimate

Posted on by
19

In my post a couple weeks back, I expressed some puzzlement about the source of the widely-quoted estimate that corruption costs the global economy approximately $2.6 trillion, or roughly 5% of global GDP. I was hoping that someone out there in GAB Reader-Land would be able to point me to the source for this figure (as several GAB readers helpfully did when I expressed similar puzzlement last year about the source for the related estimate that there are approximately $1 trillion in annual bribe payments). Alas, although several people made some very insightful comments (some of which are in the public comment thread with the original post), this time it seems that nobody out there has been able to point me to a definitive source.

I’ve done a bit more poking around (with the help of GAB readers and contributors), and here’s my best guess as to where the $2.6 trillion/5% of GDP number comes from: Continue reading →

The Amount of Bribery and the Cost of Bribery Are Not the Same

Posted on by
1

I’ve posted before (see here, here, and here) about some of my concerns regarding the accuracy of the estimates people sometimes throw around about the total amount of bribes paid each year (sometimes given in absolute terms, sometimes as a percentage of global GDP, but in all cases based on dubious extrapolations from suspect data). But for the moment I want to put those concerns aside to make another point: Even if we knew the total amount of bribes paid, that would not necessarily tell us much of anything about how much bribery costs society. (And that’s true even if we limited attention to economic costs, narrowly construed.) This is not an original point – lots of people have made it, and indeed it’s fairly obvious when you stop to think about it. Yet I keep seeing references to estimates of the amount of bribery that treat these figures as if they were measures of the cost of bribery. (For examples, see here, here, here, here, and here.) But that’s just not right. Continue reading →