Guest Post: The Problem With Anticorruption Diagnostic Tools Is Not (Primarily) Too Much Standardization

José-Miguel Bello y Villarino, an official with the Spanish Ministry of Foreign Affairs and doctoral candidate at the University of Sydney, contributes today’s guest post:

There is a wide debate about how to produce and use data to assess and compare countries’ performance, particularly in domains that are, by nature, global such as human rights. In the corruption domain there are some well-known international indexes that purport to express a country’s perceived corruption level in a single number, such as the Corruption Perceptions Index (CPI) published annually by Transparency International (TI). Other diagnostic tools have been developed to assess individual countries’ anticorruption frameworks and policies against some global standard or benchmark. Among the latter, TI produces the National Integrity System (NIS) Country Assessments.

These assessments do not try to determine how much corruption there is in a country, but rather “how well a country tackles the problem.” NIS assessments do not aim to give each country a final “score” that can be compared to the scores of other countries. The assessments’ declared objective is to look into the effectiveness of each country’s anticorruption institutions by focusing on a standard set of “pillars” (things like democratic institutions, the judiciary , the media, and civil society). Consequently, NIS assessments are not meant to provide definitive conclusions, but rather observations within a common framework to supply a starting point for analysis, and to identify risks and possible areas for improvement. Their conclusions are designed to help stakeholders work to develop more concrete and country-specific responses.

The NIS Country Assessments, and similar tools (TI has identifies roughly 500 diagnostic tools used in the anticorruption area), have come in for a fair share of criticism. Much of this criticism centers upon their allegedly formalistic, formulaic, standardized approach to assessing anticorruption institutions. Some of those criticisms have appeared on this blog. A few months ago Richard Messick posted a commentary on a piece by Paul Heywood and Elizabeth Johnson that challenged the relevance and value of NIS reports for developing democracies (using Cambodia as an illustrative example), principally due to insufficient appreciation of cultural distinctiveness and an overemphasis on compliance-based approaches. Last month, Alan Doig’s post continued this conversation. Mr. Doig defended the value of the NIS Country Assessments as they were originally conceived, but argued that TI’s current approach to NIS assessments has become overly formalistic, which limits the utility of NIS country studies as an effective starting point for analysis or platform for progression. Though coming from a different perspective, Mr. Doig’s criticism is very similar to the core argument of Professors Heywood and Johnson. In essence, they share a skepticism that one can usefully apply broad global standards or categories to individual countries, given each country’s unique, particular, idiosyncratic circumstances.

Respectfully, I think these criticisms go too far. Taking individual country circumstances into consideration of course has value. However, standardization of assessment methodologies, the somewhat “formulaic” approach, can have benefits that may outweigh the costs. Continue reading

Assessing Corruption Assessments: TI’s National Integrity System

Paul Heywood and Elizabeth Johnson raise important questions in a recent journal article about Transparency International’s corruption assessment methodology; their article deserves close attention by consumers and producers of any type of corruption assessment.  The purpose of a corruption assessment is to determine where a country is falling short in the fight against corruption and what more it needs to do.  An assessment is the backbone of any national anticorruption policy, providing both a roadmap for reform and a gauge for measuring progress, and with a wrong map or inaccurate gauge, the chances the policy will curb corruption are slight.

TI calls its corruption assessment method the National Integrity System (NIS).  One of the more than 500 different corruption assessment methodologies (or “tools” in anticorruption jargon) now in use, it is among the oldest and most widely used.  Since 2001, it has been an input into anticorruption policy in over 100 countries.  Heywood and Johnson find it has four weaknesses  – Continue reading