As regular readers of this blog know, I’ve been (figuratively) pounding my fists on the table for a while now about various misuses and misinterpretations of Transparency International’s Corruption Perceptions Index (CPI), particularly in the context of misleading year-to-year comparisons (see here, here, here, here, here, and here). Perhaps I’m overemphasizing a relatively small issue, but it seems that the problem just won’t go away.
Case in point: A piece in last Friday’s New York Times by Carol Giacomo – a member of the New York Times’ editorial board – on recent developments in Indonesia. Most of the piece is a perfectly fine discussion of recent troubling events involving conflict between the Indonesian anticorruption agency (the KPK) and the Indonesian police. But near the end, in discussing the broader implications of recent events for anticorruption efforts in Indonesia, Ms. Giacomo writes:
Transparency International, which annually rates countries on corruption in their public sectors, says Indonesia has improved its performance on the organization’s “corruption perception index” from 1.9 in 2003 to 34 in 2014[.]
Almost everything about that statement is flawed.
- First, Transparency International has not (so far as I can tell) made any statement about how Indonesia’s performance on the CPI has improved between 2003 and 2014; the only link Ms. Giacomo provides is to TI’s general 2014 CPI results page. TI has made some statements about countries whose CPI performance has improved or worsened (and I’ve criticized those statements), but those recent statements have not included any commentary on Indonesia.
- Second, TI changed its methodology for calculating CPI scores in 2012, and to try to make sure no one did exactly what Ms. Giacomo did – compare pre-2012 scores to post-2012 scores – TI actually changed the scale from a 0-10 scale to a 0-100 scale. So when Ms. Giacomo says Indonesia’s TI score improved from 1.9 to 34, and makes this seem like a jump of 32.1 points, she’s actually comparing two entirely different scales without seeming to realize it. The last time I caught this same error in a prominent outlet, the piece had been written by an intern, so I tried not to be too harsh and left the author’s name out of my post. But in this case we’re talking about a prominent journalist who really ought to know better.
- Third, and in connection with the previous point, TI itself would not want Indonesia’s 2003 score to be compared to Indonesia’s score in any other year, because as TI has acknowledged very publicly, pre-2012 CPI scores are not comparable over time.
- Finally, as I’ve tried to point out in several recent posts (see here, here, and here), post-2012 CPI scores are not comparable across time either, though failure to recognize that point is more understandable, as my critique is fairly recent and has not (yet) been widely recognized or acknowledged by TI itself.
None of this matters all that much to the main substance of Ms. Giacomo’s piece – which of course raises the question of why she bothered to mention the change in CPI scores at all. Maybe there has been an improvement on perceptions of corruption in Indonesia, and maybe much of that change is due to the KPK. Indeed, I happen to believe both things are true. But not because of a meaningless comparison between Indonesia’s 2014 and 2003 CPI scores. Really, an outlet that considers itself the paper of record ought to do better on things like this.