We’ve spent a fair amount of time, in the early days of this blog, talking about the challenges of measuring corruption cross-nationally. The well-known perception measures are useful to a point, but suffer from well-known drawbacks, chief among them concerns about how accurately perceptions capture reality. A recent working paper by Laarni Escresa and Lucio Picci, “A New Cross-National Measure of Corruption,” tries to get around these difficulties. Using data on enforcement of foreign anti-bribery laws like the U.S. Foreign Corrupt Practices Act (FCPA), Escresa and Picci they derive a new index, which they call the Public Administration Corruption Index (PACI), to make more objective cross-country comparisons in corruption levels. The paper is really clever and creative—but in the end I think it doesn’t work. Let me first say what I think is so cool about the idea, and then explain what I think are the biggest flaws.
Let’s start with the problem Escresa and Picci set out to solve. We’d like to use objective data (not subjective perceptions) to measure corruption. National law enforcement statistics are objective (at least when we can believe the numbers), but they are likely to be influenced not only by actual corruption levels, but also differences in substantive law, enforcement strategies, and court systems. For that reason, we can’t directly compare, say, domestic bribery convictions in Nigeria and Indonesia to figure out whether there is more bribery in Nigeria or Indonesia. If Indonesia has more domestic bribery convictions than Nigeria, maybe it’s because Indonesia has more corruption, of because Indonesia enforces its anticorruption laws more aggressively, or because the standard of proof is lower in Indonesian courts, or any number of other things. For enforcement data to tell us something about corruption levels, we need to find a way to control for the legal system (substantive law, prosecution, adjudication, etc.). Escresa and Picci have hit on a super-clever way to do that: Instead of looking at domestic anti-bribery laws, they look at enforcement of foreign anti-bribery laws, like the U.S. Foreign Corrupt Practices Act (FCPA).
Here’s the basic idea, simplified a bit: Suppose firms potentially subject to U.S. FCPA jurisdiction engage in the same amount and type of business activity in Nigeria and Indonesia. But suppose further that it turned out that there were twice as many FCPA actions involving bribes paid to Nigerian government officials as Indonesian officials. From this, we might reasonably conclude, based on objective data, that corruption is a more serious problem in Nigeria than Indonesia. After all, other characteristics of the enforcer (the U.S.) are constant across the two cases (same law, same prosecuting agencies, same court system), and by assumption the amount and type of business activity is the same. Under those assumptions, one would expect that, if the amount of corruption were the same, then the number of FCPA actions involving each country’s public officials would be the same on average (with some random noise).
One obvious difficulty with this approach is that the degree of FCPA risk in different countries will depend on how many FCPA-risk transactions take place in each country. If there are lots more FCPA actions involving Mexico than, say, Bangladesh, this doesn’t necessarily mean that Mexico is more corrupt than Bangladesh; it might just mean that more firms subject to FCPA jurisdiction are doing business in Mexico than in Bangladesh. Escresa and Picci are well aware of this, and try to control for this using total U.S. exports to each country to establish a baseline expected level of FCPA actions. So, if bribery were equally common in Mexico and Bangladesh, but the U.S. exports five times as much to Mexico as to Bangladesh, then Escresa and Picci predict we would observe (on average) five times more FCPA actions involving Mexico, compared to Bangladesh. If, on the other hand, we observed only, say, three times as many Mexico FCPA actions, that would suggest bribery is actually more common in Bangladesh than Mexico, once we’ve controlled for trade.
The method the paper uses to construct the PACI index is a bit more complicated (it takes into account, among other things, enforcement by multiple jurisdictions, not just the U.S.), but the above should give you the gist. As I said at the top, I think this is a really creative idea, and I highly recommend the paper. That said, let me highlight two key assumptions of the analysis—about which Escresa and Picci are themselves fully and commendably transparent—which I think are likely to be violated in practice, which compromises the validity, and therefore the utility, of the index they construct.
First, the paper assumes that controlling for exports from the enforcing jurisdiction to the host jurisdiction adequately controls for FCPA (or equivalent) risk that is due to the quantity or type of transactions, rather than the prevalence of corruption in the host jurisdiction. There are several reasons that assumption might not hold:
- FCPA liability requires bribery of a foreign public official, and even though that term is construed broadly, potential FCPA exposure may depend on the degree of state ownership in the host country economy. Imagine a pharmaceutical company paying bribes to hospitals two countries, and suppose further that the probability of a government official, or a hospital official, demanding or accepting bribes is constant across the two countries. If one country’s hospitals are state-run, and the other’s are private, the former will likely have more FCPA risk, even though corruption is identical in both. And note that this introduces a particular kind of bias into the PACI measure: countries with large state sectors will appear more corrupt, relative to countries with small state sectors.
- Exports are not the only activity that raises FCPA risk. So does foreign direct investment, for example. FDI might not show up in export data. If that’s right, Escresa and Picci’s method will tend to understate baseline predicted corruption in those countries where firms subject to FCPA jurisdiction invest heavily, but don’t export much, compared to those countries where firms export a lot. Again, the concern is not just about accuracy, but also about bias: the mix of FDI and exports to different jurisdictions might differ according to certain systematic factors (for example, very poor countries may purchase fewer U.S. exports but attract more U.S. FDI, and the PACI would tend to overstate corruption in those countries).
- Exports are likely to be affected by corruption levels (that is, to use the jargon, exports are “endogenous”). More importantly, they may be affected in different ways. For example, firms may be more reluctant to pull out of, or to avoid particularly risky transactions in, big markets (e.g., China) or markets with scarce natural resources (e.g. Nigeria). The PACI might therefore indicate corruption is higher in those places, compared to other, smaller or less special markets, where high levels of corruption will have a bigger negative impact on exports.
Second, the paper assumes that enforcing jurisdiction’s enforcement strategy does not focus more on some countries than on others, for reasons not captured by the total type/quantity of activity or by the amount of corruption in the host jurisdiction. But that assumption is questionable:
- U.S. enforcers (or others) might scrutinize more aggressively transactions in countries thought to be “high risk” than other countries. This is relevant in particular to the tests Escresa and Picci run to show the strong correlation between the PACI and the Transparency International Corruption Perception Index (CPI). But suppose U.S. enforcers target transactions countries with worse CPI scores. That would tend to induce a correlation with the PACI, even if the CPI was not picking up true differences in objective corruption levels.
- There might other reasons, besides differences in true (or perceived) corruption risk for an enforcing jurisdiction to target transactions in certain countries rather than others. Those might include political reasons, or reasons related to the relative costs of investigation, or the likelihood of effective cooperation with host jurisdictions, or the ease of proof. If these differences are random, the problem is less serious, though it still might undermine the accuracy of the indicators. But if these considerations are systematically correlated with other factors—which they might well be—then this would raise questions about use of the PACI to make causal inferences.
Now, I haven’t said anything that Escresa and Picci aren’t aware of. Indeed, in a model of transparency and candor, they discuss the above objections, and several others, in the body of the paper. I just don’t think I buy their answers. Rather than extending what has already been an overlong post, I’ll encourage anyone who’s interested to read their paper and decide for themselves how well they respond to the above critiques.