On Theory, Data, and Academic Malpractice in Anticorruption Research

I’m committed (probably self-servingly) to the idea that academic research is vital to both understanding and ameliorating corruption. I sometimes worry, though, that we in the research community don’t always live up to our highest ideals. Case in point: A little while back I recently asked my law school’s library to help me track down some research papers on corruption-related topics, including a working paper from a few years ago, co-authored by a very well-known and influential corruption/good-governance researcher. I’d seen the paper cited in other articles but couldn’t find it. The library staff couldn’t find it either, and emailed the authors directly to ask if a copy of the paper was available. Here is a verbatim reproduction of this famous professor’s response:

Thanks for your email. Unfortunately, we decided not to finish this paper since we could not get the data to fit our theory[.]

I have to say, I found this response a bit troubling.

Now, to be fair, maybe what this person (whose first language is not English) actually meant was that he and his coauthor were unable to locate the data that would allow a meaningful test of the theory. (In other words, perhaps the statement “We could not get the data to fit our theory” should be understood to mean: “We could not acquire the sort of data that would be necessary to test our theory.”) But boy, much as I want to be charitable, it sure sounds like what this person meant was that he and his coauthor had tried to slice and dice the data in lots of different ways to get a result that fit a predetermined theory (so-called “Procrustean data torturing”), and that when they couldn’t get nature to confess, they spiked the paper rather than publicizing the null findings (contributing to the so-called “file drawer problem”).

Now, again, maybe that latter reading is wrong and unfair. Maybe the more charitable interpretation is actually the correct one. But still, it’s worrying. Even if this case was not, in fact, itself an illustration of the data torturing and the file-drawer problem, I’m sure those things go on in anticorruption research, just as they do elsewhere. Lots of scholars (including the author of the above email) have their own pet theories about the best way to promote high-quality governance, and spend quite a bit of time advising governments and NGO reformers on the basis of these (allegedly) evidence-based theories. But for the results of academic research to be credible and useful, we all need to be very careful about how we go about producing our scholarship, and to be careful not to let our findings — or our decisions about what projects to pursue, publish, and publicize — be unduly determined by our preconceived notions.

The Roles of Anticorruption Academics and Advocates: Insights from the NGO Side

One of the purposes of this blog (as noted in our mission statement) is to promote the interchange of ideas across disciplinary boundaries, including–indeed, especially–between researchers and practitioners. It turns out that despite our shared interests in understanding and fighting corruption, there’s often quite a gulf between the academic and advocacy communities. I’ve commented this difference in perspectives in the past (from the perspective of an Ivory Tower academic), both in general terms, and with respect to some particular topics, such as the optimal degree of simplification, the role of university education, and the use of eye-catching statistics. While I recognize that discussion of these issues may seem like navel-gazing, I actually think these conversations are quite important, given the complementary but distinct roles that academic research and advocacy work have in the overall anticorruption project.

I was therefore delighted to read a recent speech by Robert Barrington, the Executive Director of Transparency International UK, on precisely this topic. It’s one of the best discussions of this issue that I’ve come across. (And I’d say that even if he didn’t reference one of my posts on this blog!) Whereas I come at this issue from an academic perspective, Mr. Barrington is a leading voice in the advocacy community, and he has some good advice for all of us. The speech is very short, so instead of attempting to summarize it I’ll just encourage interested readers to click on the link above. But let me close here by quoting Mr. Barrington’s summation, with which I wholeheartedly concur:

We should be two communities that work closely together. There is little excuse not to. As an advocate, this is my message: our subject is too important for academics to be obscure or self-referential, or for NGOs to be ill-informed, misguided or unchallenged. Our choice is not whether to work hand-in-hand, but how we should do so.