Guest Post: Using Open Data To Combat Corruption—Moving Beyond the Hype

Robert Palmer, the Director of Partnerships and Communication at the Open Data Charter, contributes today’s guest post:

In order to tackle corruption effectively, one first needs to understand the networks that link government officials, businesses, and professional intermediaries, and then work to either dismantle these networks or at least ensure that these webs of connections are not exploited to enrich individuals and undermine good government. Fortunately, these clandestine networks often leave traces in government-held databases, such as company registers, land title deeds, asset disclosures, and other official records. That’s where open data can be helpful. When the government provides easily accessible public information, it makes it easier for government officials, journalists, and citizens to follow financial flows, understand who’s providing government services, and to spot suspect behavior. And that’s why there has been so much enthusiasm about the open data in the anticorruption community. In 2015, for example, the G20 anticorruption working group announced a common approach saying that “Open Data can help prevent, detect, investigate and reduce corruption.”

Yet what’s happening on the ground isn’t living up to this hype. Part of the reason is that, as the Web Foundation and Transparency International found in a recent study of five G20 countries, many countries have made only limited progress toward meeting international commitments on open data. But even where open data is available, relatively few organizations are actually using open data to expose and combat corruption. There are, of course, exceptions, including Global Witness, the data journalists at Organised Crime and Corruption Reporting project, and accountability groups such as BudgIT. Yet the potential for open data to help fight corruption remains largely unrealized.

To help address this shortcoming, the Open Data Charter has spent the last year pulling together a guide for how to use open data to combat corruption. The guide lists 30 types of datasets that could help expose and combat corruption if they are released in the right way, as well as key data standards to ensure consistency and quality between different countries. Of course, the underlying assumptions here are that the types of data listed in the guide can be collected and released by governments in the ways the guide advises, and that there are anticorruption actors who can process this data in ways that are helpful in exposing or preventing corruption. In order to probe these assumptions, the Open Data Charter has teamed up with the Government of Mexico to “road-test” the guide. This will include working out which of the 30 datasets in our guide the government already publishes, which further ones can be released, and how to engage potential users. We’re interested in understanding how if data is released in the right way, users such as journalists, law enforcement, and civil society can process the data and then use it to have an impact on corruption.

Our approach to this piece of work is guided by a real desire to learn what works: what’s helpful to the government and what’s helpful to external stakeholders who want to tackle corruption. We hope to be able to report on our initial findings over August. If you’re interested in learning more, please get in touch with me: robert [at] In the spirit of transparency and collaboration, the guide itself is open to comment here.

Guest Post: Turning Big Data Into a Useful Anticorruption Tool in Africa

GAB is delighted to welcome back Dr. Elizabeth Dávid-Barrett of the University of Sussex, who contributes today’s guest post:

Many anticorruption advocates are excited about the prospects that “big data” will help detect and deter graft and other forms of malfeasance. As part of a project in this vein, titled Curbing Corruption in Development Aid-Funded Procurement, Mihály Fazekas, Olli Hellmann, and I have collected contract-level data on how aid money from three major donors is spent through national procurement systems; our dataset comprises more than half a million contracts and stretching back almost 20 years. But good data alone isn’t enough. To be useful, there must be a group of interested and informed users, who have both the tools and the skills to analyse the data to uncover misconduct, and then lobby governments and donors to listen to and act on the findings. The analysis of big datasets to find evidence of corruption – for example, the method developed by Mihály Fazekas to identify “red flags” of corruption risks in procurement contract data—requires statistical skills and software, both of which are in short supply in many parts of the developing world, such as sub-Saharan Africa.

Yet some ambitious recent initiatives are trying to address this problem. Lately I’ve had the privilege to be involved in one such initiative, led by Oxford mathematician Balázs Szendrői, that helps empower a group of young African mathematicians to analyse “big data” on public corruption. Continue reading

More Phony Numbers–This Time on the Anticorruption Impact of Open Data

OK, I know I’m beating a dead horse. Within the last month I’ve already posted several times (see here, here, and here) about bogus anticorruption statistics, as has Rick. And I promise that after this post, I’ll move on to other topics. But I can’t help commenting on this latest release from Transparency International, criticizing the recent World Economic Forum (WEF) meeting for not explicitly addressing corruption. As its lead example, TI faults the WEF for not addressing issues like open data (and openness more generally). I’m sympathetic to TI’s policy position, but in making the case, TI asserts, “One study suggests that open data could reduce the costs of corruption by about 10 percent.”

I was curious (and, admittedly, skeptical) about yet another seemingly precise estimate of something that’s inherently hard to measure. So I clicked on the link to the “one study” that “suggests” that open data technologies would reduce the costs of corruption by 10%. This “study” is actually a report (really, an advocacy document) from an Australian consulting firm (Lateral Economics), commissioned by a philanthropic fund (the Omidyar Network) that invests in open data initiatives. How does this “study” reach its conclusion that open data could reduce the costs of corruption by 10%? I will now quote in full the entirety of the evidence and analysis supporting that conclusion: Continue reading

Shedding Sunlight on Procurement

In a previous post, I extolled the virtues of Big Data in the fight against corruption, including in the important realm of government procurement. From the UK to Georgia to the Czech Republic, government procurement agencies have been collaborating with civil society groups to analyze their data, uncovering inefficiencies that range from the mundane to the outright corrupt. Governments are not alone: international development agencies like the World Bank are embarking on similar projects.

But there’s a problem. Big Data needs lots of data to work, entailing a high degree of government transparency and massive disclosures — sometimes called Open Government — that are sometimes at odds with the goals of anticorruption. In the case of government procurement, public data watchers need to know which firms bid for the project, at what price, and who won on what terms before they can play a useful watchdog role. However, as Rick has pointed out on this blog, public disclosure rules in procurement has the perverse effect of enabling private collusion. Cartels of contractors can agree amongst themselves to inflate their prices and select which among them will receive the contract, and are able to enforce their shady agreement because, of course, all offers are public.

Rick’s concerns seem to be directly implicated by the newly-proposed Open Contracting Data Standard, a push to “enhance and promote disclosure and participation in public contracting.” The project essentially asks every procurement agency in the world to upload their contracting documents onto the internet in a standardized manner that would encourage public oversight, including through the use of Big Data tools. So, is the push for open government procurement data doomed to backfire, creating collusion where perhaps it did not even exist before? Fortunately not. The increased risk of collusion is completely outweighed by the potential for the use of Big Data and other civil society monitoring techniques. Continue reading