Like many people out there, I’m both a huge fan of Nate Silver–and the rigorous quantitative approach to election forecasting that he popularized–and at the same time quite disappointed in his FiveThirtyEight website, where the posts (especially those not by Silver himself) often seem to be slapdash efforts by people who have a smattering of statistical knowledge but don’t really know much about the topics they’re writing about. A depressing recent example, germane to this blog, is a post from last week entitled “It Only Seems Like Politics Is More Corrupt.” I normally wouldn’t bother to comment on something so slight here (especially because the post appears to have been written by an intern, and I generally try to avoid beating up on people who are just starting out), but many of the errors in analysis are both sufficiently elementary, and sufficiently common in discussions of corruption trends in other contexts (and by people with much more experience and therefore less of an excuse), that it’s worth taking a moment to explain what’s wrong.

A quick summary: The author cites recent U.S. Gallup poll data showing that the percentage of Americans who believe that “corruption is widespread” throughout the government in the United States has increased from about 60% in 2006 to a little over 75% in 2013. However, the author argues, the data doesn’t support the idea that corruption in the U.S. has actually worsened. To support that claim, she points to two other data sources:

1. U.S. Department of Justice statistics from 1992-2012 show that the number of cases prosecuted by the DOJ’s Public Integrity Section (as well as the number of convictions and number of cases awaiting trial) appears to have declined, or at least hasn’t increased.
2. The U.S. score on the Transparency International Corruption Perception Index (CPI) hasn’t changed very much between 1995 and 2013 (although there’s concededly a slight downward trend).

Do these two data sources disprove the idea that corruption in the U.S. has worsened over the last eight years, or more generally that the U.S. public’s perception of corruption is inaccurate?  In a word, no. There are so many elementary conceptual and statistical errors in this analysis, it’s difficult to know where to begin, but let me take a shot at cataloguing the most egregious problems: Continue reading →