Tag Archives: instrumental variables

Dear People Doing Quantitative Research on Corruption: Please, Please, Please Stop Using Clearly Invalid Instrumental Variables.

Posted on by
2

I will open this post with two apologies: First, this is going to be on a (seemingly) nerdy and technical subject (though one that non-technical folks who read statistical papers on corruption really need to understand). Second, this post is going to return to a subject that I wrote about two years ago, without adding much, except perhaps different examples and somewhat more intemperate language. But the issue is an important one, and one that I think needs more attention, both from the people who produce quantitative empirical studies on corruption and those who consume those studies.

The issue concerns a particular statistical technique sometimes deployed to figure out whether variable X (say, absence of corruption) causes variable Y (say, economic growth), when it’s possible that the correlation between X and Y might arise because Y causes X (or because some third factor, Z, causes both X and Y). The technique is to find an “instrumental variable” (an IV for short). To be valid, the IV must be sufficiently correlated with X, but could not conceivably have any affect on Y except through the IV’s casual effect on X. The actual estimation techniques used in most cases (usually something called “two-stage least squares”) involve some additional statistical gymnastics that I won’t get into here, but to get the intuition, it might help to think about it this way: If your instrumental variable (IV) correlates with your outcome variable (Y), and there’s no plausible way that your IV could possibly affect Y except by affecting your proposed explanatory variable (X), which then has an effect on Y, then you can be more confident that X causes Y. But for this to work, you have to be very sure that the IV couldn’t possibly affect Y except through X. This assumption cannot be tested statistically–it can only be evaluated through common sense and subject-area expertise.

OK, if you’ve slogged your way through that last paragraph, you may be wondering why this is important for corruption research, and why I’m so exercised about it. Here’s the problem: Continue reading →

Invalid Instrumental Variables in Corruption Research: A Lament

Posted on by
6

A while back, I posted a critical commentary on Paulo Mauro’s widely-cited paper purporting to show that corruption lowers foreign investment and growth. My criticisms focused on Mauro’s use of a statistical technique called “instrumental variables” (or “IV”) analysis, which — when done properly — can help figure out whether a hypothesized explanatory variable actually causes an outcome of interest, or whether instead the observed statistical correlation is due to the fact that the alleged outcome variable actually influences the proposed explanatory variable (“endogeneity” or “reverse causation”).  But an IV analysis requires making certain strong and untestable assumptions about the relationships between the variables.  If those assumptions are wrong, the conclusions one draws about causation will be unsound (not necessarily wrong, but not worthy of credence on the basis of the analysis).

This may seem like an issue that only stats nerds should care about, but I actually think it’s important that other researchers, activists, and policy advisers understand the basics of the technique and how it can go wrong (or be misused).  I say this because a surprisingly large amount of the research on the causes and consequences of corruption — research that is often cited, individually or collectively, in discussions of what to do about corruption — relies on this technique. And, I hate to say it, but much of that research uses IV analysis that is clearly inappropriate.

I’ve been thinking about this issue recently because I’ve been going through the literature on the relationship between democracy and corruption for a paper I’m writing, and this issue crops up a lot in that literature. But I’ve seen essentially the same problems in lots of other research on corruption’s causes and consequences, so I’m reasonably confident that this is not an isolated problem.

Let me say a bit more about the essence of the statistical problem, how IV analysis is supposed to solve it, and why much of the IV analysis I’ve seen (focusing on the democracy-corruption context) is not worthy of credence: Continue reading →