In my last post, I discussed my unsuccessful attempts to track down the source for the widely-cited “$1 trillion in annual bribe payments” figure (other than a 2004 World Bank press release, which referenced an unpublished study without further citation). Several readers were kind enough to direct me to the best published source on the $1 trillion figure: the appendix in a chapter by Daniel Kaufmann in the World Economic Forum’s 2005-2006 Global Competitiveness Report. The chapter addresses most—though perhaps not all—of my concerns.
As to the methodology, Kaufmann’s appendix explains that the strategy was to use existing surveys that asked business for the amount that their firm (or firms like theirs) paid each year as a percentage of sales, as well as surveys that asked households about annual bribe payments as a percentage of household income. These surveys included the 2000 World Bank Enterprise Survey (WBES) (covering 81 countries) and the World Economic Forum’s 2004 Global Competitiveness Survey (104 countries), as well as individual World Bank surveys carried out in several different countries. The researchers then extrapolated to countries not covered in the surveys, and produced a range of estimates under different assumptions and scenarios.
The main concern I raised in my earlier post was the opacity of the methodology used to generate the $1 trillion number; that issue is largely addressed by Kaufmann’s appendix, which – though perhaps not detailed enough to enable replication by an outside researcher – is sufficient to understand the process used to generate the numbers.
My other big complaint about the widely cited press release is that it provided a misleadingly-precise point estimate, rather than a range of plausible values that reflects the inherent uncertainty in the data. Kaufmann’s chapter does not suffer from that problem. Indeed, the methodology he describes is commendable in its exploration of a large number of scenarios, in order to generate a range of possible amounts for annual bribe payments, which, Kaufmann reports, could be as low as $600 million or as high as $1.5 trillion (or more). His concluding paragraph frames this in exactly the right way:
“In summary, based on this exercise, a reasonable range of estimates for annual bribery would appear to be between US$0.6 and well over US$1.5 trillion a year, with a reasonable midpoint being close to US$1 trillion.”
So the main problem here is not so much the original study (though I have some lingering questions about the data reliability and the extrapolation techniques, and I do think it’s not ideal that the actual study has not, to my knowledge, been made available in a form that would allow researchers to attempt replication). The bigger concern is with how this admirably cautious and qualified attempt to quantify annual bribery got translated into the oft-repeated and misleadingly precise “$1 trillion in annual bribes”. I know it’s not that big a deal – after all, as I said in the original post, I doubt anyone’s take on the bribery problem would be that different if the number were $500 million or $1.5 billion. But I do think it’s important to be as clear as possible about what we know and what we don’t know in this area, as in others. Indeed, to my mind the more important punch-line from Kaufmann’s research is not the $1 trillion figure, but rather the magnitude of the uncertainty — which drives home just how little we know, and how hard it is to figure this stuff out.
GAB | The Global Anticorruption Blog 
Interesting post that calls attention to a problematic assumption. After glancing through Kaufmann’s appendix, I noticed that he incorporated data from as early as 1999. Given that Kaufmann’s estimate partially relies on data that is now roughly 15 years old, do you think that this is another concern serious enough to doubt the annual bribes figure? I would assume that increased globalization and advances in telecommunications has made bribery easier thus indicating the possibility that the annual figure is higher or closer to 1.5 trillion dollars.
Additionally, how often do you think these types of studies are needed? I think that another study incorporating more recent data is definitely needed.
I’m sure the estimate is out of date, though whether the total amounts have gone up or down, I have no idea. A lot has happened in the last 15 years.
As to your second question: My basic answer to the question “how often are these types of studies needed?” is “never.” We know there’s lots of bribery, and we know that these estimates are always going to be hugely imprecise. And every second that smart people like Daniel Kaufmann and his team devote to estimating total annual bribe payments is a second that could have been used more productively on something else. Now, if I thought these numbers could actually tell us whether global bribery was trending up or down, then I’d say, OK, maybe studies like this might be useful. But given how broad the estimated range is in Kaufmann’s original study, it’s hard to imagine we could ever discern trends at conventional levels of statistical significance. So my instinct is to say, don’t bother.
All that said, I grudgingly recognize that for political/advocacy/media purposes, it might sometimes be useful to come up with a (misleadingly) hard number to convey the scale of the activity in question. I guess. Maybe.
One weakness (fatal flaw?) in the bribe estimate numbers is that they are derived from answers to a question posed to firms in different countries along the lines of: “How much do firms in your industry pay in bribes”? What reasons are there to think the answers are reliable? Valid?
Sometimes those reporting the data hint that the question is posed with a sly wink or a nod so that the respondent realizes he or she is being asked about how much his or her firm paid. For some reason this does not increase my confidence in either the reliability or validity of the resulting data.
There might have been a time when estimates like the $1 billion, with or without the range, was useful for advocacy purposes. Might. But today? Is it not time for estimates that are reliable and valid? Even if we can’t measure the level of bribery in a nation (let alone the whole world?) we can develop estimates for particular sectors in a nation– health, construction, education. And almost invariably one of the “positive externalities” of such work is ideas about how to reduce the incidence of bribery.