One of the big challenges in anticorruption work, which I suspect we will be discussing quite a bit on this blog, concerns the measurement of corruption. After all, there are a bunch of different theories about the causes and consequences of corruption, and about the best way to combat it. Testing these theories requires some way of measuring the extent of corruption (or different forms of the corruption problem). And for folks actually doing anticorruption work (donors, governments, NGOs, etc.), it would be nice to be able to assess how well programs are working. Yet all of the existing measures have significant problems.
To try to inspire some creative thinking about new ways to measure corruption, the good people at the U4 Anti-Corruption Resources Centre (affiliated with the Christian Michelsen Institute in Bergen, Norway), with the assistance of the UK’S Department for International Development (DFID), recently sponsored a competition (the “Proxy Challenge”) to come up with new proxies that would help track the progress of anticorruption reform initiatives. U4 hosted a one-day workshop last month to let the five finalists present their proxies, to choose a winner, and to promote some general discussion of the challenges of developing useful proxies for corruption in a variety of contexts. I was able to attend. I’ll try to post a some more substantive thoughts in a later post, but here are a few quick reactions.
First, I think this competition was a great idea, and I hope that U4 and DFID consider doing it again–perhaps advertising it more broadly, and maybe attaching a substantial award, rather than the lovely but largely ceremonial prize sculpture that the winners received (I think it was some kind of Norwegian duck sculpture). My sense is that this first attempt at running a Proxy Challenge was a bit disappointing: there were only a small number of total entries, and even some of the finalists didn’t really introduce fresh or creative approaches to measuring corruption, but rather deployed fairly conventional project evaluation methods (number of complaints successfully addressed, increases in government revenue, etc.) in new contexts.
Second, although not all the entries were equally strong, the winning entry, from Mihály Fazekas and István János Tóth of the Corruption Research Center Budapest (CRCB), is well worth a look, particularly for those working on corruption in public procurement. I won’t try to summarize the methodology here, but—long story short—the look at a set of 13 “risk factors” for corruption in public contracting (identified by their strong correlation with suspicious public contracts, such as those where one company always seems to win or where there’s very little actual competition despite the fact that there are several viable competitors out there), and then use those factors, weighted by their significance, to construct a “corruption risk index.” There’s more to it as well, but I’ll leave it to interested readers to check out the presentation itself, as well as the associated paper. (Two related CRCB working papers are available here and here.)
Finally, one thing that became quite obvious over the course of the discussions was the enormous gulf between the academic/researcher types and the policy/donor/program types on what makes a proxy valuable. I may do a separate post specifically on this topic, but it was quite striking how frustrated the practitioners seemed to get with the academics’ (perceived) preference for complex, data intensive measures, and at the same time how frustrated the academics got with the practitioner group’s (perceived) attraction to simple (crude?), easy-to-summarize metrics. Given that both groups seem genuinely committed to combating corruption, the cultural chasm was striking. But kudos to U4 for holding this event and attempting to bridge this gap.
Great to see you blogging on the Proxy Challenge. Having co-orchestrated this initiative I feel like contributing to your blog post, hoping that the debate on corruption measurement will continue.
First of all, the rationale for the initiative was twofold. First, we wanted to make the case that to advance from the current plateau we need to move focus beyond standardised, aggregate, cross-country corruption indices, and ways to measure bribery and financial fraud. Instead, we should explore how bespoke indicators work in specific country contexts, working bottom-up rather than top-down. Second, we were hoping to find some indicators that our partners could use at the country level to evaluate performance on control of corruption. We always know that entries would be “rough gems.” In fact, we saw little value in trying yet again to find the “one indicator to rule them all.” None of the proposals found the sweet spot balancing elegance with robustness, but they all contributed to advancing the frontier on measurement.
My sense is that the initiative has contributed to re-igniting the debate around the first issue. Your blog will hopefully help and sustain some of that momentum. Essentially, we need to realise that there are no easy fixes. Just like in other sector we need to do the bread-and-butter work if we want good measurement tools. On the second issue, it is true that the initiative did not produce a set of readily applicable new proxy indicators. But, for me at least, that was never realistic to expect. We do hope that U4 partner agencies, and/or other aid agencies, are interested in working on fine-tuning some of the proposals. Most of the proposals were work-in-progress. I think it would be worth supporting the continuation of this work.
There may be value in having another Proxy Challenge. I think it did help bridge the gap to some extent between practitioners and academics. Practitioners know the standard indices, but need some way to be introduced to other indicators. It also clarified that although we all talk about indicators, the real challenges often lie in issues of methods and data. Technically speaking, indicators are not always the challenge. Personally, I think that making a contribution to improving the field should be a sufficient reward. A decorative bird (not duck) is just a bonus. The workshop held in February should (hopefully) not be the end of the initiative. We are planning to work with several of the teams to refine their proposals and publish them as U4 Briefs. The ultimate aim, and reward, would be to have some of the indicators used at the country level, potentially bundled together with other indicators in a basket. Hopefully we can raise enough interest and resources to do this.
We produced a website with all the materials from the event, including video: http://www.u4.no/articles/the-proxy-workshop/