We’ve spent a fair amount of time, in the early days of this blog, talking about the challenges of measuring corruption cross-nationally. The well-known perception measures are useful to a point, but suffer from well-known drawbacks, chief among them concerns about how accurately perceptions capture reality. A recent working paper by Laarni Escresa and Lucio Picci, “A New Cross-National Measure of Corruption,” tries to get around these difficulties. Using data on enforcement of foreign anti-bribery laws like the U.S. Foreign Corrupt Practices Act (FCPA), Escresa and Picci they derive a new index, which they call the Public Administration Corruption Index (PACI), to make more objective cross-country comparisons in corruption levels. The paper is really clever and creative—but in the end I think it doesn’t work. Let me first say what I think is so cool about the idea, and then explain what I think are the biggest flaws.
Let’s start with the problem Escresa and Picci set out to solve. We’d like to use objective data (not subjective perceptions) to measure corruption. National law enforcement statistics are objective (at least when we can believe the numbers), but they are likely to be influenced not only by actual corruption levels, but also differences in substantive law, enforcement strategies, and court systems. For that reason, we can’t directly compare, say, domestic bribery convictions in Nigeria and Indonesia to figure out whether there is more bribery in Nigeria or Indonesia. If Indonesia has more domestic bribery convictions than Nigeria, maybe it’s because Indonesia has more corruption, of because Indonesia enforces its anticorruption laws more aggressively, or because the standard of proof is lower in Indonesian courts, or any number of other things. For enforcement data to tell us something about corruption levels, we need to find a way to control for the legal system (substantive law, prosecution, adjudication, etc.). Escresa and Picci have hit on a super-clever way to do that: Instead of looking at domestic anti-bribery laws, they look at enforcement of foreign anti-bribery laws, like the U.S. Foreign Corrupt Practices Act (FCPA).
Here’s the basic idea, simplified a bit: Suppose firms potentially subject to U.S. FCPA jurisdiction engage in the same amount and type of business activity in Nigeria and Indonesia. But suppose further that it turned out that there were twice as many FCPA actions involving bribes paid to Nigerian government officials as Indonesian officials. From this, we might reasonably conclude, based on objective data, that corruption is a more serious problem in Nigeria than Indonesia. After all, other characteristics of the enforcer (the U.S.) are constant across the two cases (same law, same prosecuting agencies, same court system), and by assumption the amount and type of business activity is the same. Under those assumptions, one would expect that, if the amount of corruption were the same, then the number of FCPA actions involving each country’s public officials would be the same on average (with some random noise).
One obvious difficulty with this approach is that the degree of FCPA risk in different countries will depend on how many FCPA-risk transactions take place in each country. If there are lots more FCPA actions involving Mexico than, say, Bangladesh, this doesn’t necessarily mean that Mexico is more corrupt than Bangladesh; it might just mean that more firms subject to FCPA jurisdiction are doing business in Mexico than in Bangladesh. Escresa and Picci are well aware of this, and try to control for this using total U.S. exports to each country to establish a baseline expected level of FCPA actions. So, if bribery were equally common in Mexico and Bangladesh, but the U.S. exports five times as much to Mexico as to Bangladesh, then Escresa and Picci predict we would observe (on average) five times more FCPA actions involving Mexico, compared to Bangladesh. If, on the other hand, we observed only, say, three times as many Mexico FCPA actions, that would suggest bribery is actually more common in Bangladesh than Mexico, once we’ve controlled for trade.
The method the paper uses to construct the PACI index is a bit more complicated (it takes into account, among other things, enforcement by multiple jurisdictions, not just the U.S.), but the above should give you the gist. As I said at the top, I think this is a really creative idea, and I highly recommend the paper. That said, let me highlight two key assumptions of the analysis—about which Escresa and Picci are themselves fully and commendably transparent—which I think are likely to be violated in practice, which compromises the validity, and therefore the utility, of the index they construct.
First, the paper assumes that controlling for exports from the enforcing jurisdiction to the host jurisdiction adequately controls for FCPA (or equivalent) risk that is due to the quantity or type of transactions, rather than the prevalence of corruption in the host jurisdiction. There are several reasons that assumption might not hold:
- FCPA liability requires bribery of a foreign public official, and even though that term is construed broadly, potential FCPA exposure may depend on the degree of state ownership in the host country economy. Imagine a pharmaceutical company paying bribes to hospitals two countries, and suppose further that the probability of a government official, or a hospital official, demanding or accepting bribes is constant across the two countries. If one country’s hospitals are state-run, and the other’s are private, the former will likely have more FCPA risk, even though corruption is identical in both. And note that this introduces a particular kind of bias into the PACI measure: countries with large state sectors will appear more corrupt, relative to countries with small state sectors.
- Exports are not the only activity that raises FCPA risk. So does foreign direct investment, for example. FDI might not show up in export data. If that’s right, Escresa and Picci’s method will tend to understate baseline predicted corruption in those countries where firms subject to FCPA jurisdiction invest heavily, but don’t export much, compared to those countries where firms export a lot. Again, the concern is not just about accuracy, but also about bias: the mix of FDI and exports to different jurisdictions might differ according to certain systematic factors (for example, very poor countries may purchase fewer U.S. exports but attract more U.S. FDI, and the PACI would tend to overstate corruption in those countries).
- Exports are likely to be affected by corruption levels (that is, to use the jargon, exports are “endogenous”). More importantly, they may be affected in different ways. For example, firms may be more reluctant to pull out of, or to avoid particularly risky transactions in, big markets (e.g., China) or markets with scarce natural resources (e.g. Nigeria). The PACI might therefore indicate corruption is higher in those places, compared to other, smaller or less special markets, where high levels of corruption will have a bigger negative impact on exports.
Second, the paper assumes that enforcing jurisdiction’s enforcement strategy does not focus more on some countries than on others, for reasons not captured by the total type/quantity of activity or by the amount of corruption in the host jurisdiction. But that assumption is questionable:
- U.S. enforcers (or others) might scrutinize more aggressively transactions in countries thought to be “high risk” than other countries. This is relevant in particular to the tests Escresa and Picci run to show the strong correlation between the PACI and the Transparency International Corruption Perception Index (CPI). But suppose U.S. enforcers target transactions countries with worse CPI scores. That would tend to induce a correlation with the PACI, even if the CPI was not picking up true differences in objective corruption levels.
- There might other reasons, besides differences in true (or perceived) corruption risk for an enforcing jurisdiction to target transactions in certain countries rather than others. Those might include political reasons, or reasons related to the relative costs of investigation, or the likelihood of effective cooperation with host jurisdictions, or the ease of proof. If these differences are random, the problem is less serious, though it still might undermine the accuracy of the indicators. But if these considerations are systematically correlated with other factors—which they might well be—then this would raise questions about use of the PACI to make causal inferences.
Now, I haven’t said anything that Escresa and Picci aren’t aware of. Indeed, in a model of transparency and candor, they discuss the above objections, and several others, in the body of the paper. I just don’t think I buy their answers. Rather than extending what has already been an overlong post, I’ll encourage anyone who’s interested to read their paper and decide for themselves how well they respond to the above critiques.
GAB | The Global Anticorruption Blog 
Matthew — this is a terrific post, but I just want to build on a couple things you said:
1. Unfortunately, there doesn’t seem to be a quick fix to the bias toward countries with larger state sectors — the U.S. does not prosecute transnational bribery of private actors to nearly the same degree as transnational bribery of foreign public officials, likely depriving us of the data set necessary to correct for this imbalance across countries. (Perhaps they could correct using some measure of the relative size of the state sectors in each state?) To add a further complication to the mix: the effect of the “state sector size” bias might well differ by country. It isn’t clear exactly who, if anyone, counts as “foreign official” in a state instrumentality; this means that prosecutors may (i) focus on particular types of state-owned enterprises (SOE) with which they are familiar, meaning that countries with a higher prevalence of those types of SOE (e.g., 100% state-owned corporations) might be viewed as more corrupt; or (ii) push the “instrumentality” line further in countries with which they are more familiar (because they have an easier time getting evidence to prove state-control and ownership or just have a better understanding of how state enterprises in that country work).
2. U.S. enforcement authorities routinely scrutinize some countries more closely than others, which certainly undermines the second assumption, as you noted. For example, the DOJ and SEC may ask multinational corporations to focus their internal investigations on the countries that are believed to present the biggest risk, or that have the largest state sector, or that house the corporations’ largest overseas operations. This would lead to a bias in favor of countries (i) that are already perceived to be corrupt, (ii) that have large state sectors, and (iii) that attract greater foreign business activity. U.S. enforcement authorities may also focus on particular countries because they have better relationships with those countries’ regulators and can expect to get greater cooperation, quicker MLAT responses, etc. Differences in local privacy laws could also theoretically influence the decision about where to look. Behind all of these examples is a simple notion: prosecutors are human — they want the biggest bang for their buck. As a result, they will look where they are likely to find the biggest bribery at the lowest cost, which introduces a systemic bias into any cross-national corruption index that relies on FCPA enforcement numbers alone.
3. There is another problem with the use of FCPA enforcement actions: the ratio between actual FCPA violations and FCPA enforcement actions is likely different for many countries because it is just harder to prove violations in some countries than others. Why? Well, each country presents unique obstacles in terms of evidence collection; perhaps because it is more difficult to circumvent the nation’s privacy laws or the country is less responsive than others to MLAT requests and letters rogatory. If it is harder to collect evidence regarding transactions in a particular country, prosecutors will likely be more reluctant to charge conduct involving that same country.
Jordan, I think you raise great points. For your second point, I think current FCPA enforcement trends can undermine the second assumption of the article as noted by Matthew. Your “simple notion” does have much weight– I think the fact that prosecutors want the biggest bang for their buck is further supported by companies being more likely to settle during an FCPA investigation. Because these settlement rates are favorable to the DOJ and SEC, I would not be surprised if these entities prioritized cases involving countries that are more cooperative and where there are higher concentrations of foreign investment by companies under FCPA jurisdiction.
First of all I’d like to thank you for your comments on our paper. There, we consider most of the issues that you raise. Here, let me just sketch an overall answer, more than anything, to try to situate our proposal (the “PACI”) within the debate on how to best measure corruption.
What we propose is an “objective” measure – actually, to the best of my knowledge, the first cross-national such measure (*). I hope that readers will judge it while also keeping in mind the shortcomings of the available alternatives. Also, I believe that the comparison of the PACI with currently available perception-based measures is instructive. For example, in the paper we argue that such a comparison allows us to “discover” the measurement scale of perception-based measures such as the World Bank Corruption Control Index, or Transparency International Corruption Control Index.
On the first series of issues raised by Matthew, in the paper we make the point that those “problematic” factors could be controlled for in various ways. So for example, we could proxy the number of bilateral transactions also using FDI flows (to do so, we’d need to know whether each corruption case is trade- or FDI-related – an information which currently we do not have, but which could be retrieved in principle). In general, several types of adjustments could be made to address possible departures from the hypotheses on which the validity of the PACI rests. However, executing these improvements would require a further data collection effort, within a project that already has been quite labor-intensive (at least, for the standards of academia, and given the stringent budget constraints that we face). “Apologetic-mode off”, but really, assembling the dataset as it stands was quite a lengthy ordeal.
Possible improvements could also address the second set of issues. So for example, while available evidence suggests that the effect of corruption in diverting trade flows is present, but is not very relevant, certainly our measure could be adjusted to factor any such effect out. It is also certainly possible that enforcers, as suggested, “might scrutinize more aggressively transactions in countries thought to be “high risk” than other countries”, and that this might explain the observed correlation of the (log) of the PACI with current perception-based measures. This possibility also could be studied, for example by distinguishing cases according to how they emerge. Many of them, for example, follow whistleblowing activities of various types, and maybe we may assume that enforcers act on such occurrences in ways which do not depend on the foreign country involved. But certainly, this also is an empirical question, and it could possibly be addressed. I believe that one advantage of the PACI is that it is clear in defining concepts and is transparent what may go wrong (with respect to “catch all” perception-based measures, at least), in this way facilitating trouble shooting and future improvements.
One last caveat (also as a partial answer to Jordan’s comments): the PACI is computed from cases enforced not just in the US, but everywhere. The effect of any bias which may be present, unless there is a positive perfect correlation across different jurisdictions, would be dampened in the aggregate.
Again, thank you for your comments, to which, I realize, my short answer does not do full justice. Anyway, either here or in private, I’d be glad to continue this conversation (my email address is listed at the top of my Web page – see below).
Best,
Lucio Picci
University of Bologna
(*) In the past, with a different co-author, I proposed one such objective measure, which may be computed at the sub-national level. See: http://www2.dse.unibo.it/picci/article.pdf
Lucio,
Thanks for the thoughtful response. In case it wasn’t clear enough in my original post (which focused mostly on my criticisms and reservations), I think this is a great project, and represents a significant advance over the usual reliance on perceptions-based data. I do remain skeptical that the current version of the PACI — which, one must keep in mind, is in its first iteration — succeeds in providing an objective, unbiased estimate of underlying corruption levels, and might be particularly problematic if used to address certain kinds of research questions (where the biases I worry about may distort the results). But my criticisms were offered in the spirit of trying to push the larger project forward, to address these concerns. I hope you and others continue to work on this (and that you get a bigger budget to do so!).
(By the way, I think you may not be totally right that this is the first attempt to construct an index that measures corruption objectively — see my post tomorrow for discussion of some other attempts.)
I encourage all our readers out there to check out the paper. Notwithstanding my reservations, it’s an important contribution.
Greetings Lucio, from the south Pacific,
I am interested to note the omission of New Zealand from your tables. Was there any particular reason(s) for this?
Given that New Zealand has featured at the top or first equal of the TI CPI, for quite some time, I would have thought that it’s inclusion would be telling for the overall results?
Not enough data? Too small? Too many sheep?
More seriously, as a journalist / commentator on widespread and systemic failure of regulatory oversight there (and also of TI CPI and such like), I am genuinely intrigued.