OK, in my post from a few weeks back, I asserted that year-to-year changes in a country’s Corruption Perceptions Index (CPI) score are not meaningful, even after the thoughtful and welcome changes that Transparency International made to its methodology in 2012. My concern was — and remains — that the underlying data sources that TI uses to create the index are themselves not likely to be comparable across years, which means that the CPI inherits the problem. But for purposes of this post, I’m going to completely disregard my own warning in that earlier post, and take a look at whether there have been fact been any notable changes in individual countries’ perceived corruption between 2012 and 2013.  Based on a very quick scan of the data, the answer appears to be (mostly) no.

Continue reading →

In post last week, I emphasized what lots of others have already tried (without much apparent success) to point out: Prior to 2012, Transparency International’s Corruption Perception Index (CPI) scores are not comparable over time.  The fact that a country’s score from one year to the next goes up or down might reflect an actual change in perceived corruption, but might be due to a whole host of other factors (changing aggregation methodology, changing scope of country coverage, change in perceived corruption of other countries, etc.), such that simple year-to-year comparisons are unreliable.  In making this point, I was not criticizing TI itself, which has been quite clear that the pre-2012 CPI scores cannot be compared across years.

But what about 2012 and after?  In 2012, TI announced, with much fanfare, that starting with the 2012 CPI and henceforth, scores across years would be comparable, due to changes in methodology (described here).

Is this right?  If it were, it would be a huge benefit both to scholars and policy reformers who want to evaluate changes over time–and the impact of various interventions.  Alas, after reading through TI’s discussion of the revised methodology, I regret to say the answer is probably no.

Continue reading →

OK, I know (as Rick pointed out in a recent post) that a lot — maybe too much — of the content on this blog has focused on measurement issues, so I apologize for yet another post on that topic, but this has really been bugging me:

Transparency International has been publishing its well-known and widely-used Corruption Perceptions Index (CPI) since 1995.  The index has its pros and cons, several of which have been discussed on this blog (see here, here, here, here, and here).  But putting other debates about the CPI’s validity and utility to the side, one thing should be perfectly clear: At least prior to 2012 (when TI changed its method and scoring system for the CPI), a country’s CPI scores CANNOT be compared across years.  The fact that Country X scores, say, a 4.4 in 2002, and scores a 4.9 in 2005, does NOT mean that (perceived) corruption has declined in country X.  Maybe it did, but it might have stayed the same, or gotten worse.  At most, the pre-2012 CPI provides information about country’s ranking relative to other countries, within a single year, with respect to corruption perceptions.

TI itself could not be more explicit about this, stating bluntly “CPI scores before 2012 are not comparable over time.”  Yet I keep coming across sources — news articles, presentations by leading international organizations, academic papers — that use year-to-year CPI comparisons to make claims about how corruption in a particular country or region is improving or worsening, or about whether a particular policy intervention is working or not.  YOU CAN’T DO THIS!  PLEASE STOP!!

Continue reading →