Perception-based corruption indicators, though still the most widely-used and widely-discussed measures of corruption at the country level, get a lot of criticism (some of it misguided, but much of it fair). The main alternative measures of corruption include experience surveys, which ask a representative random sample of firms or citizens about their experience with bribery. Corruption experience surveys are neither new nor rare, but they’re getting more attention these days as researchers and advocates look for more “objective” ways of assessing corruption levels and monitoring progress. Indeed, although some early discussions of measurement of progress toward the Sustainable Development Goals (SDGs) anticorruption target (Target 16.5) suggested—much to my chagrin—that changes in Transparency International’s Corruption Perceptions Index (CPI) score would be the main measure of progress, more recent discussions appear to indicate that in fact progress toward Goal Target 16.5 will be assessed using experience surveys (see here and here).

Of course, corruption experience surveys have their own problems. Most obviously, they typically only measure a fairly narrow form of corruption (usually petty bribery). Also, there’s always the risk that respondents won’t answer truthfully. There’s actually been quite a bit of interesting recent research on that latter concern, which Rick discussed a while back and that I might post about more at some point. But for now, I want to put that problem aside to focus on a different challenge for bribery experience surveys: When presenting or interpreting the results of those surveys, should one control for the amount of contact the respondents have with government officials? Or should one focus on overall rates of bribery, without regard for whether or how frequently respondents interacted with the government?

To make this a bit more concrete, imagine two towns, A and B, each with 1,000 inhabitants. Suppose we survey every resident of both towns and we ask them two questions: First, within the past 12 months, have you had any contact with a government official? Second, if the answer to the first question was yes, did the government official demand a bribe? In Town A, 200 of the residents had contact with a government official, and of these 200, 100 of them reported that the government official they encountered solicited a bribe. In Town B, 800 residents had contact with a government official, and of these 800, 200 reported that the official solicited a bribe. If we don’t control for contact, we would say that bribery experience rates are twice as high in Town B (20%) as in Town A (10%). If we do control for contact, we would say that bribery experience rates were twice as high in Town A (50%) as in Town B (25%). In which town is bribery a bigger problem? In which one are the public officials more corrupt?

The answer is not at all obvious; both controlling for contact and not controlling for contact have potentially significant problems:

With respect to reporting corruption experience results without controlling for contact, the big problem is that citizen-government interactions may be higher or lower for reasons that have nothing to do with the integrity of the government per se (that is, the number of interactions may be uncorrelated with the probability that any individual interaction will involve bribery). If that is so, then a jurisdiction with more citizen-government interactions will appear to have more bribery, even if in fact its public officials are no more dishonest—or even if they are actually more honest—than in some other jurisdiction with fewer government-citizen interactions.

• For example, suppose that in our hypothetical Town A and Town B, the reason that four times as many citizens in Town B have had contact with the government is that the government of Town B recently introduced a program to distribute free mosquito nets, and this program served 600 people. In the absence of the program, 200 of Town B’s residents would have had contact with the some other government officials (say, the local business licensing bureau), and 100 of them would have been solicited for bribes—just like in Town A. There’s some corruption in Town B’s mosquito net program, but it’s actually lower than for the business licensing bureau; only one in six citizens who go to get their free mosquito net are hit up for a bribe. But because this program reaches so many more people, the total number of Town B’s citizens who end up paying brines is higher than in Town A. Doesn’t it seem a little odd to suggest that the bribery problem is worse in Town B than in Town A? We can make the point even stronger if we imagine that the reason Town A decided not to introduce a similar mosquito net program was that Town A’s government predicted that bribery rates in the program would be close to 100%. This is obviously a stylized “toy” example, but the broader point here is that the size and nature of government programs may correlate with the overall bribery experience rate, independently of the integrity of the government officials.
• In the above example, the difference in contact rates was “supply-side” driven: more government programs means more bribery, even if government integrity remains constant. A similar problem can arise when the frequency of government contacts is driven by changes on the “demand side.” Suppose, for example, that in both Town A and Town B, the only government agency at issue is the business licensing bureau. And suppose further that the reason contact rates in Town B are so much higher is that Town B is experiencing an economic boom, and many citizens are either starting new businesses or expanding their existing businesses, and as a result need to apply for a business license. Most people would reject the conclusion that Town B’s officials are more corrupt than officials in Town A, even though there’s more bribery in Town B.

We might naturally conclude that if the big problem with looking at overall bribery rates is that frequency of contact with the government could vary for reasons that had nothing to do with the government’s integrity, the solution is to control for bureaucratic contact frequencies. If we do, then in the above examples, we would conclude that government integrity is lower in Town A than in Town B, even though more of Town B’s citizens pay bribes. However in other situations controlling for frequency of contact with the government will produce misleading results, because the frequency of contact is itself affected by the integrity of government officials. As before, there are both supply-side and demand-side versions of this problem:

• Starting with the demand-side, citizens may avoid contact with government officials whom they expect will demand bribes, and as a result, those citizens who are most likely to be victimized may end up not interacting with government officials at all. To see this, consider a modified version of the example with Town A and Town B. Suppose here that in both Town A and Town B, 200 citizens interact with the local business licensing bureau, and in both towns 100 of those citizens are solicited for bribes. Now assume that both Town A and Town B introduce the mosquito net distribution program. Assume that in Town A, the program reaches 600 citizens, 300 of whom are solicited for bribes. In Town B, by contrast, the local officials in the relevant office are widely known by the citizens to be thoroughly corrupt, such that the bribes they would demand exceed the ability of the citizens to pay; as a result, no citizens in Town B bother trying to get their mosquito nets (which, we might imagine, are ultimately misappropriated by the officials and sold at a profit). If we control for bureaucratic contact when interpreting the results of bribery experience data, we would conclude that bribery is equally prevalent in both towns, as in both cases 50% of the citizens who had contact with a public official (400 out of 800 in Town A, 100 out of 200 in Town B). But in this admittedly artificial example, that’s clearly the wrong inference, as corruption in Town B is clearly worse, and the rates of bribery are equal, when controlling for contact, only because citizens in Town B deliberately avoid contact with those government officials who are especially likely to demand bribes.
• The supply-side version of the same basic problem is that sometimes the government may deliberately expand (or in some cases limit) the frequency of government-citizen interactions in order to increase bribery opportunities. But bribery opportunities may rise at a decreasing rate as government activity increases, especially if the government deliberately starts with the most promising bribe opportunities. To illustrate this, suppose that in both Town A and Town B, all of the government-citizen interactions involve government inspectors showing up at the respondent’s business to make sure all the paperwork is in order; if the inspector finds any irregularity, no matter how minor, she can demand a bribe to overlook it. (For the sake of this example, let’s assume that the inspector can’t fabricate a violation completely.) Suppose now that the reason Town B has so many more government-citizen interactions is that Town B is the inspectors deliberately try to increase their bribe revenues by conducting four times as many inspections. The percentage of inspections that result in a bribe is lower because the first 200 they look at are the ones that are the most likely targets. Nevertheless, in this example the higher overall bribe frequencies in Town B are due to more predatory, bribe-seeking behavior, such that it would seem strange to assert that the bureaucracy is twice as honest in Town B as in Town A. But that’s the conclusion you would draw if you were to use bribery experience surveys and control for contact.

I wish I could offer a constructive suggestion about what to do about these problems, but I’m still unsure. It should go without saying that both corrected and uncorrected bribery experience rates can and should be presented, but that’s not so helpful for people trying to figure out how to assess progress toward corruption targets. My instinct is that in most cases it makes more sense to correct for frequency of contact, because I think that concerns that contact frequency may vary for reasons that have little to do with government integrity are probably more significant than concerns that contact frequency may be strongly correlated with government integrity. But that’s just a hunch; I can’t offer anything like a compelling argument in favor of that view. At the very least, as bribery experience surveys are increasingly used to measure progress toward anticorruption targets, this question needs to become a more central part of the discussion.

One other point that it might be worth noting here before I close: In some cases, the main results of bribery experience surveys will be presented as follows:  “Of all the respondents who had at least one contact with a public official, what proportion were solicited for a bribe?” That might look like it controls for frequency of bureaucratic contacts, but it doesn’t, at least not completely or sufficiently. Intuitively, the possibilities of a bribe solicitation are higher when there are more and more frequent contacts. Controlling only for respondents who have a single contact of any kind doesn’t really get at the fundamental concern.