Last month, we had a spirited debate in the anticorruption blogosphere about the conceptualization of corruption, academic approaches to the study of the topic, and the relationship between research and practice. (The debate was prompted by provocative piece by Bo Rothstein, to which I replied; my critical reaction prompted a sur-reply from Professor Rothstein, which was followed by further contributions from Robert Barrington, Paul Heywood, and Michael Johnston.) I’ve been thinking a bit more about one small aspect of that stimulating exchange: How do we, or should we, think about evaluating the success (or lack thereof) of an anticorruption policy or other intervention? I was struck by the very different assessments that several of the participants in last month’s exchange had regarding whether the anticorruption reform movement had been “successful,” and this got me thinking that although part of the divergence of opinion might be due to different interpretations of the evidence, part of what’s going on might be different understandings of what “success” does or should mean in this context.
That observation, in turn, connected to another issue that’s been gnawing at me for a while, that I’ve been having trouble putting into words—but I’m going to take a stab at it in this post. My sense is that when it comes to defining and measuring “success” in the context of anticorruption reform (and probably many other contexts too), there’s a fundamental tension between two conflicting impulses:
On the one hand, anticorruption advocates would like to promote an evidence-based reform strategy, one that prioritizes interventions that can be rigorously assessed—if not quantitatively, then at least through reliable qualitative evidence based on expert or participant observation. Emphasizing rigorous monitoring, evaluation, and testing of “what works” (as practitioners sometimes put it) helps ensure that that those supporting anticorruption reform efforts can invest their resources most effectively and efficiently. This approach is also helpful in building political and institutional support for anticorruption reform efforts, because advocates can point to interventions that have a proven track record.
On the other hand, there is a widespread understanding that, at least in those societies where corruption is a systemic problem, fighting corruption is a long-term process, one that requires fundamental changes in the political, social, and economic order. These changes can take decades, or even generations, to accomplish. This is not to say that policy interventions or institutional reforms are irrelevant, and that the transition from endemic corruption to norms of integrity, when it occurs, is due to “deep” processes of social evolution that policymakers cannot affect. In fact, “good government” reforms of various types may be a critical part of the process. But the process takes so long that the effects of even an extremely important and successful reform might not be felt for years after adoption. Moreover, the process of political and societal transformation is sufficiently complex that it may be impossible to isolate and trace the impact of any individual reform.
But of course this gives rise to a challenge for advocates and reformers, as well as for researchers trying to evaluate the success or failure of various approaches to anticorruption. Contrary to the periodic assertions that all anticorruption interventions have been abject failures (or the more moderate criticism that we still have no idea of “what works” when it comes to fighting corruption), in fact we do have a fair amount of evidence on the short-term impact of specific interventions. (The most obvious example that leaps to mind concerns various forms of financial oversight, such as public expenditure tracking and regular audits. We also have qualitative reports of how certain interventions have changed behavior on the ground, or how legal reforms have altered incentives in ways that discourage bribery.) But this evidence is sometimes dismissed, or at least downplayed, because these incremental changes, which often involve more low-level transactional corruption, do not indicate meaningful changes in the larger order and/or the prevailing norms of governance.
That’s a fair point. But when we turn to that larger question, and ask what sorts of interventions might help—and what can learn from past experience—it turns out that it’s extremely hard to say anything with great confidence. (Side note: Distrust anyone who does say anything with great confidence about how societies and norms of governance transform. Scholars have some educated guesses and intriguing theories, but on this subject extreme intellectual humility is essential.) I can’t stress enough that this does not mean that reformers should not pursue this larger transformative agenda. In my view, it is vitally important that they do so. And researchers (academics and others) have a crucial role to play here, maybe even more so than in the context of shorter-term project assessments, because in the absence of reliable observable evidence about the effects of a given intervention, we must rely even more on theory, by which I mean simplified mental models of how the world works that can provide a coherent and parsimonious account of important known facts, and that can generate plausible hypotheses and predictions that can provide some guidance, however provisional and imperfect, to practitioners. But in this context “success” is virtually impossible to assess rigorously, at least not with our existing tools.
What follows from all this? I’m not quite sure, but let me offer a few concluding thoughts.
- First, I do think that at least some of the disagreement over whether anticorruption interventions have been successful or unsuccessful arises because people have different understandings of “success” in mind, and end up talking past one another.
- Second, I think (returning to my above observation about theory) that different people have different unspoken mental models of the relationship between short-term, measurable, incremental progress and long-term systemic change. My conjecture (and it is only that!) is that some people implicitly assume that every little bit helps, and that small but concrete improvements are both substantively meaningful and will cumulatively contribute to larger transformations in governance norms; others, by contrast, tend to see the sorts of positive changes attributable to discrete interventions as marginal at best, and as having little to do with the bigger-picture struggle for systemic change. So, when presented with a study showing that a new auditing system for local governments in Country X (where corruption is reputed to be rampant) resulted in a 12.6% reduction of missing funds, one person would be inclined to celebrate this as an important success and evidence that meaningful progress is possible in Country X, while another person would think that this is mostly beside the point when we know most high-level politicians in Country X still embezzle substantial sums and elections largely financed with off-the-books illicit donations from wealthy private interests who expect and receive favorable government treatment in return. Which of these views is closer to the truth is itself an empirical question, one to which we do not yet have a definitive answer (and to which the answer may depend on a host of contextual factors).
- Third, I tend to think that both perspectives have something useful to offer, and that whichever way we lean, we should try to make an effort to see things from the other perspective as well. Those who are more practically-minded and concerned with measurable on-the-ground improvements must avoid falling into the trap of thinking that if something isn’t measurable it isn’t important, or of losing sight of the bigger picture, or of neglecting the relevance to anticorruption of changes that are not on their face directly about corruption per se. And those who tend to think in bigger-picture terms must take care not to neglect or dismiss the cumulative importance of seemingly marginal changes, must stay appropriately humble about our collective ignorance regarding macro-systemic change, and must be mindful of the political and institutional imperatives that may drive a focus on measurable short-term outputs and outcomes.
- Fourth, we could all stand to be a bit clearer in explaining what we mean by “success” in this context. It’s a bit funny to me that the anticorruption community spends so much time and energy (an excessive amount, in my view) arguing about the correct conceptualization, definition, and measurement of “corruption,” yet seems to pay comparatively little attention to explicitly discussing the proper conceptualization, definition, and measurement of “success,” as applied to any given anticorruption intervention, or the anticorruption movement as a whole.
Those are just a few preliminary thoughts. I’d welcome further thoughts—and vigorous disagreement—from readers.
I enjoyed reading your piece and it struck a chord with me. It is a question I grabble with in my current role as a cabinet minister with policy making responsibilities. The observations you make about conflicting impulses is real. What should anticorruption success look like? As policy makers, we want to showcase evidence of success, otherwise we risk irrelevance. On the other hand, we would like to bring transformational changes but take cognizance of the fact that it is extremely difficult to supplant normative structures in society so we trade off with small scale interventions that we hope will count in the long run. We do not even have the evidence that this works in the long term but we trudge on. I can relate with the frustration of anticorruption interventions and the sentiments of those who assert that all anticorruption interventions have been abject failures. Many people would like to see big time corruption end. Just like many people would like to see racism end, at the stroke of a pen. Or gender discrimination end. So, I agree with you and think part of this frustration comes from our own expectations of what success should look like and societal expectations of what that success should look like, which all differ for all us, practitioners, academia and even members of the public.
As a policy maker, I focus more on small wins, no matter how short term. This is purely a survivalist tactic, as any attempt to tackle systemic corruption is extremely dangerous. This is the reality of the dangers of studying corruption in the third world. However, what I have found working albeit slowly, is public education. Empowering the people with information gives them the tools to demand accountability from leaders. This has its downside as leaders do not always slow down on their corrupt behavior but instead engage in massive coverups to avoid public scrutiny. Practitioners do not know where to push the levers to make things work and obtaining empirical data is critical. However, in my country obtaining such data is difficult, as corruption research is not exactly welcome. I had difficulties getting a supervisor for my PhD, and real drama in the field as potential respondents literally ran off. I am now using part of my data to write about corruption methodologies, particularly in a third world context and countries where corruption is endemic.
I strongly disagree with you that academia should stop spending too much time conceptualizing, defining and measuring corruption, my reason being that, knowledge never ends, and we need to add to what we already know. More importantly, if we don’t know what it is, how can we know what success will look like when we eradicate it? I think the two are inter related. It is important we define it because if we don’t know what it is or can’t even agree on what it is, how will we know when we eradicate it? How can we even begin to discuss it if we don’t know what color it is, who has seen it? Do we assume our eyes will recognize it when we see it and our ears will hear it when it makes a sound (assuming it makes a sound and assuming it is something we can see). I think we need to engage more in theorizing it, I think academicians are leaning too much towards practice and forgetting to develop theories or abandoning theory making, they should provide the theoretical conceptualizations (with input from practitioners), as both need to work together. Corruption is a wicked problem with so many dimensions and any one intervention has so many implications that it requires rigorous interrogation.
But there is one point I do agree with you, that perhaps it’s time we gave more attention to conceptualizing corruption success, as that will find more immediate application for us in public policy making. Although I must point out here that there are assumptions that efforts to fight corruption are ideologically noble and rational. Such moves have been known to facilitate globalization that benefits Western countries. Further, there are serious ethical issues around corruption, for example in matters of life and death, where a family member needs emergency treatment and there is no way to get it unless one bribes the doctor or bribes to obtain life-saving equipment. Even at the international level, there are ethical issues faced by international aid organizations when they are forced to give bribes to militia or government officials to be allowed to supply essential services to disaster-stricken families. Currently, there are scholars who argue that corruption is indeed desirable and promotes efficiency in government by greasing the bureaucratic wheels so that they turn faster for those seeking public services. They argue that corruption emerges when government interventions that are designed to deal with some distortion in the market system, fail, as people try to beat the government system. They also posit that corruption in the form of gift-giving fulfills the function of creating trust in business relationships. Even the practice of striving for a complete reduction of possibilities of corruption in public procurement, amidst the constant threat of disfavored suppliers raising their voice against allegedly corrupt decisions, has led to public procurers basing their decisions on the accurate price of the service procured, rather than the riskier, but ultimately more effective concept of the most economically valuable service. In such contexts, fighting corruption can be dysfunctional and contrary to the public good. So yes, let’s have a debate on what anticorruption success looks like!
Thanks for this comment; I really appreciate the perspective of an actual policy practitioner here. That said, I have to express some skepticism over your last paragraph on situations in which fighting corruption can be contrary to the public good. To use your example about the family member who must bribe a doctor in order to receive life-saving treatment, I don’t think anyone would advocate for cracking down on the bribe-paying family member, but rather fighting corruption in such a way that makes it not necessary to pay a bribe in order to receive life saving treatment. Similarly, I don’t think that anticorruption efforts are aimed at condemning the aid organizations that pay bribes to access disaster-stricken families, but rather at preventing the militias and governments from demanding bribes in the first place.
Anticorruption efforts, to my mind, exist on a spectrum of policy options and targets, not merely a binary of “condemning all bribery versus not fighting corruption at all.” I think that anticorruption advocates can easily choose not to crack down on those who pay bribes in situations where they have no choice (while still going after those who demand the bribes) without claiming that “[i]n such contexts, fighting corruption can be dysfunctional and contrary to the public good.”
Your perspectives have enriched this discussion. I agree with you about fighting corruption in such a way that one never has to bribe to get life saving treatment or bribe militia to access disaster-stricken families. To ensure this does not happen, we may need to change the structure of society and this discussion becomes one of whether anticorruption efforts can successfully attempt to do so and what might success look like? I am not really for a one-size fits all solution but for practical solutions that might help in specific contexts. I agree that we do have policy options to target specific issues but these do not often work perfectly and we still do not have adequate empirical evidence on what works and why.
Pingback: Episode 277 – the Immunized Not Vaccinated edition | The Compliance Podcast Network
This discussion got me thinking, how could we as anticorruption practitioners, change the course of history? How could we rewire society? How could we change the fundamental structures in society and enable them to respond to anticorruption tools? Short of a revolution, I think certain interventions can bring this much desired change. I am thinking of disruptive technologies…