Guest Post–Assessing Corruption with Big Data

Today’s guest post is from Enestor Dos Santos, principal economist at BBVA Research.

Ascertaining the actual level of corruption is not easy, given that it is usually a clandestine activity, and much of the available data is not comparable across countries or across time. Survey data on corruption experience can be helpful, but it is often limited to very specific kinds of corruption (such as petty bribery). Researchers and analysts have therefore, quite reasonably, tended to rely on subjective corruption perception data, such as Transparency International’s well-known Corruption Perceptions Index (CPI). (The CPI aggregates corruption perception data from a variety of other sources, mostly expert assessments.) But conventional corruption perception measures (including those use to construct the CPI) have well-known problems, including limited coverage (with respect to both years and countries) and relatively low frequency (usually annual). And they rely on the perceptions of a handful of experts, which may not necessarily be representative. These limitations mean that while traditional perception measures like the CPI may be useful for some purposes, they are not as helpful for others, such as measuring the impact of individual events or news reports on corruption perceptions, or how changes in corruption perceptions affect government approval ratings.

To address these concerns, a recent study by BBVA Research, entitled Assessing Corruption with Big Data, offered an alternative, complementary type of corruption perceptions measure, based on Google web searches about corruption. To construct this index, we examined all web searches classified by Google Trends in the “Law and Government” category for individual countries, and calculated the proportion of those searches that contain the word “corruption” (in any language and including its misspellings and synonyms). Our index, which begins in 2004, covers more than 190 countries and, unlike traditional corruption indicators, is available in real-time and with high-frequency (monthly). Moreover, it can be reproduced very easily and at very low cost.

Here are some of our main findings: Continue reading

How “Scandalizing” Corruption Can Backfire

High profile corruption scandals are making headlines almost every day: Israeli Prime Minister Benjamin Netanyahu is embroiled in multiple bribery allegations; Brazilian President Luiz Inácio Lula da Silva (known as Lula) was convicted for his involvement in corruption; Peruvian President Pedro Pablo Kuczynski was forced to resign after his allies were caught on tape buying political support to defeat his impeachment vote. The list could go on and on. And one cannot help noticing that the media coverage of these high-profile corruption cases often focuses on the most lurid, sensational aspects of individual politicians’ corrupt behavior. For example, as the Netanyahu probes unfolded, the Israeli media emphasized the juicy details: how Netanyahu and his wife were bribed with Cuban cigars and Dom Pérignon worth up to $130,000, the state’s annual allocation of approximately $3,000 for the PM’s pistachio ice cream supply, and his son’s bragging of how his father pushed through a gas deal caught on tape in a strip club. And this is but one example. It seems that corruption cases are often covered as if they were TV dramas, with entertaining plot twists and voyeuristic appeal. To put this in the terminology developed by Shanto Iyengar in his book on how TV news frames political issues, much of the contemporary media coverage of corruption tends to be “episodic” (focusing on individual stories or specific events, putting the issues in a more subjective light, and including sensational or provocative content) as opposed to “thematic” (more systematic, abstract, and in-depth, and providing a wider context for a more nuanced understanding of the causes and trends).

Such salacious coverage of corruption is perhaps unavoidable; these tawdry details attract more readers and viewers than dry reporting on financial misdeeds and back-room negotiations. And one might think that such coverage would be more effective in motivating citizens to take action against corruption—whether through votes, protests, organizing, or other means. After all, as Jimmy Chalk argued last year on this blog, anticorruption narratives can be more effective when they include dramatic stories with virtuous heroes and sinister villains. That may well be true for narratives fashioned by activists in the context of a campaign, but for news reporting, the episodic/scandal-centric approach may be counterproductive, for three main reasons:

Continue reading

Is the West Being Too Critical of Corruption in Ukraine? The Debate Continues

A couple weeks back I posted a commentary on an interesting debate over the West’s approach to promoting anticorruption in Ukraine. On the one side, Adrian Karatnycky (the Managing Partner of a consulting firm that assists international clients with government relations in Ukraine) and Alexander Motyl (Professor of Political Science at Rutgers) published a piece arguing that the West’s approach to promoting anticorruption was misguided, for two reasons: First, because (according to the authors) there was too much focus on punishing individual wrongdoers rather than on institutional reform, and second because the emphasis on the failings of the Ukrainian government (and the wrongdoing of individual Ukrainian officials) was undermining a reformist government, and would likely lead Ukrainian voters to embrace populist demagogues. On the other side, Daria Kaleniuk (the executive director of a Ukrainian civil society organization called the Anti-Corruption Action Center) countered that the only reason the Ukrainian government has made any progress on anticorruption reforms is because of pressure from the West, and that holding individual wrongdoers accountable is essential to making progress on this issue and restoring the faith of the Ukrainian people in the institutions of government.

My own take was that Ms. Kaleniuk is likely correct that individual accountability, though not sufficient, is a necessary component of an effective anticorruption strategy; Karatnycky and Motyl’s implicit argument that Ukraine could make headway on the corruption problem without an effective system for holding individual wrongdoers accountable, as long as the country pursues “institutional reforms” (like privatization and de-monopolization), struck me as both facially implausible and inconsistent with what we know about successful anticorruption reforms elsewhere. Karatnycky and Motyl’s second point, about “messaging,” struck me as harder. On the one hand, it’s true that emphasizing only problems and failures and shortcomings might breed cynicism, frustration, and possibly political instability. But on the other hand, exposing corruption may be the only way (or at least the most effective way) to mobilize public opinion to address some very real problems.

I probably wouldn’t have returned to this topic (about which, I can’t repeat enough, I lack genuine expertise), but Mr. Karatnycky and Professor Motyl published a rejoinder to Ms. Kaleniuk last week that I think merits further commentary. The new piece makes a number of separate points, and I won’t touch on all of them. But if I had to sum up their central argument, it would go like this:

Don’t be too critical of the ruling elites—even if those elites are pretty corrupt, and even if the only reason they’ve done much of anything about corruption in the past is because they’ve been pressured or shamed or coerced into doing so. If you’re too mean to them, they might lose the support of the people—and what comes next might be much worse.

That summary, which I admit is a bit of a caricature, might seem unfair. But I don’t think it is. Indeed, I not only think it’s an accurate distillation of Karatnycky and Motyl’s main argument, but I actually think that it’s an argument worth taking seriously, and in some circumstances might even be right. But I’m skeptical it’s right in most cases, and I remain to be convinced that it’s right about Ukraine. Under most conditions, I think it’s probably wrongheaded and dangerous to say that we shouldn’t criticize a government for failing to tackle corruption or try to expose the corruption of individual politicians out of a concern that doing so might undermine the legitimacy of the government.

So, before I proceed, let me make clear that my caricature—“Don’t say mean things about the kinda-corrupt-but-kinda-reformist incumbents”—really is a fair distillation of the argument. Here the key passages from Karatnycky and Motyl’s most recent piece: Continue reading

Brazil: A Model for International Cooperation in Foreign Bribery Prosecutions

Much ink has been spilled celebrating the extraordinary crackdown on corruption in Brazil over the past few years (including on this blog). Headlined by the massive Operation Car Wash (Portuguese: Lava Jato)—in which officials received nearly $3 billion in bribes to overcharge Petrobras, Brazil’s state-controlled oil company, for construction and service work—high-profile corruption investigations have swept through Brazil, threatening to upend its reputation as a bastion for unchecked graft. Although corruption in Brazil remains a serious problem, the extensive investigations have worked to elevate the nation as an inspiration for countries looking to address their own corrupt political systems and hoping to become “the next Brazil.”

In addition to the headline-grabbing investigations targeting the upper echelons of the Brazilian government, Brazilian authorities have also worked closely with U.S. authorities investigating bribery activity in Brazil, leading to significant penalties both under Brazilian law and under the U.S. Foreign Corrupt Practices Act (FCPA). This is a significant development, because it demonstrates the possibility for close collaboration on cross-border bribery cases between a developed country (usually on the “supply side” of transnational bribery cases) and a developing country (on the “demand side”). Commentators have complained that too often supply-side enforcers like the United States take an outsized role in transnational bribery cases, with the countries where the bribery takes place doing too little. Other commentators have cautioned that an increase in prosecutions by other countries, in the absence of some sort of global coordination mechanism, may lead to races to prosecution or to over-enforcement. China’s nearly $500 million fine of British pharmaceutical giant GlaxoSmithKline in 2014 for bribing Chinese doctors and hospitals was emblematic of these fears, providing an example of an aggressive, unilateral approach to demand-side enforcement – while putting DOJ in the unfamiliar position of pursuing FCPA violations as a cop late to the scene.

Through its recent enforcement actions, Brazil has provided a different model. While there have been successful joint enforcement actions in the past—such as the Siemens case—the recent series of coordinated U.S.-Brazil actions exhibit how developed and developing countries can work together in anti-bribery enforcement, sharing in the investigative responsibilities, negotiations with companies, and even the financial returns.

Continue reading

More on the 2017 Corruption Perceptions Index, and the Relationship Between Media/Civil Society Freedom and Corruption

The rest of the anticorruption commentariat (and the mainstream media) may have already moved on from the publication of Transparency International’s 2017 Corruption Perception Index (CPI), but I wanted to follow up on my other posts from earlier this month (here and here) to discuss one other aspect of the new CPI. The general overview, press release, and other supporting materials that accompanied the latest CPI stress as their main theme the importance of a free press and a robust, independent civil society in the fight against corruption. As TI states succinctly in the overview page for the 2017 CPI, “[A]nalysis of the [CPI] results indicates that countries with the least protection for press and non-governmental organisations (NGOs) also tend to have the worst rates of corruption.” And from this observation, TI argues that in order to make progress in the fight against corruption, governments should “do more to encourage free speech, independent media, political dissent and an open an engaged civil society,” and should “minimize regulations on media … and ensure that journalists can work without fear of repression or violence.” (TI also suggests that international donors should consider press freedom relevant to development aid or access to international organizations, a provocative suggestion that deserves fuller exploration elsewhere.)

Speaking in broad terms, I agree with TI’s position, and I’m heartened to see TI making an effort to use the publicity associated with the release of the CPI to push for concrete improvements on a particular area of importance, rather than simply stressing the bad effects of corruption (such as the alleged adverse impacts on inequality and poverty), or devoting undue attention to (statistically meaningless) movements in country scores from previous years. Whether TI succeeded in leveraging the CPI’s publicity into more attention to the freedom of the media and civil society is another story, but the effort is commendable.

That said, I spent a bit of time digging into the supporting research documents that TI provided on this issue, and I find myself in the uncomfortable position of finding the proffered evidentiary basis for the link between a free press/civil society and progress in the fight against corruption problematic, to put it mildly—even though my own reading of the larger academic literature on the topic makes me think the ultimate conclusion is likely correct, at least in broad terms. That latter fact, coupled with my recognition that the materials I’m evaluating are advocacy documents rather than academic research papers, makes me reluctant to criticize too harshly. Nonetheless, on the logic that it’s important to hold even our friends and allies accountable, and that in the long term promoting more careful and rigorous analysis will produce both more suitable policy prescriptions and better advocacy, I’m going to lay out my main difficulties with TI’s data analysis on the press freedom-corruption connection: Continue reading

Adjusting Corruption Perception Index Scores for National Wealth

My post two weeks ago discussed Transparency International’s newly-released 2017 Corruption Perceptions Index (CPI), focusing in particular on an old hobby-horse of mine: the hazards of trying to draw substantive conclusions from year-to-year changes in any individual country’s CPI score. Today I want to continue to discuss the 2017 CPI, with attention to a different issue: the relationship between a country’s wealth and its CPI score. It’s no secret that these variables are highly correlated. Indeed, per capita GDP remains the single strongest predictor of a country’s perceived corruption level, leading some critics to suggest that the CPI doesn’t really measure perceived corruption so much as it measures wealth—penalizing poor countries by portraying them as more corrupt, when in fact their corruption may be due more to their poverty than to deficiencies in their cultures, policies, and institutions.

This criticism isn’t entirely fair. Per capita income is a strong predictor of CPI scores, but they’re far from perfectly correlated. Furthermore, even if it’s true that worse (perceived) corruption is in large measure a product of worse economic conditions, that doesn’t mean there’s a problem with the CPI as such, any more than a measure of infant mortality is flawed because it is highly correlated with per capita income. (And of course because corruption may worsen economic outcomes, the correlation between wealth and CPI scores may be a partial reflection of corruption’s impact, though I doubt there are many who think that this relationship is so strong that the causal arrow runs predominantly from corruption to national wealth rather than from national wealth to perceived corruption.)

Yet the critics do have a point: When we look at the CPI results table, we see a lot of very rich countries clustered at the top, and a lot of very poor countries clustered at the bottom. That’s fine for some purposes, but we might also be interested in seeing which countries have notably higher or lower levels of perceived corruption than we would expect, given their per capita incomes. As a crude first cut at looking into this, I merged the 2017 CPI data table with data from the World Bank on 2016 purchasing-power-adjusted per capita GDP. After dropping the countries that appeared in one dataset but not the other, I had a 167 countries. I then ran a simple regression using CPI as the outcome variable and the natural log of per capita GDP as the sole explanatory variable. (I used the natural log partly to reduce the influence of extreme income outliers, and partly on the logic that the impact of GDP on perceived corruption likely declines at very high levels of income. But I admit it’s something of an arbitrary choice and I encourage others who are interested to play around with the data using alternative functional forms and specifications.)

This single variable, ln per capita GDP, explained about half of the total variance in the data (for stats nerds, the R2 value was about 0.51), meaning that while ln per capita GDP is a very powerful explanatory variable, there’s a lot of variation in the CPI that it doesn’t explain. The more interesting question, to my mind, concerns the countries that notably outperform or underperform the CPI score that one would predict given national wealth. To look into this, I simply ranked the 167 countries in my data by the size of the residuals from the simple regression described above. Here are some of the things that I found: Continue reading

The New Corruption Perceptions Index Identifies Countries with Statistically Significant Changes in Perceived Corruption–Should We Credit the Results?

As most readers of this blog are likely aware, last month Transparency International (TI) released the 2017 edition of its important and influential Corruption Perceptions Index (CPI). As usual, the publication of the CPI triggered a fair bit of media coverage, much of it focused on how various countries ranked, and how individual country scores had changed from one year to the next (see, for example, here, here, here, and here).

There’s a lot to say about the most recent CPI—I may devote a post at some point to TI’s interesting decision to focus the press release accompanying the publication of the 2017 CPI less on the index itself than on the connection between (perceived) corruption and a lack of adequate freedom and protections for the media and civil society. But in this preliminary post, I want to take up an issue that regular GAB readers will know has been something of a fixation of mine in past years: the emphasis—in my view mostly misplaced—on how individual country CPI scores have changed from year to year.

In prior posts, I’ve raised a number of related but distinct concerns about the tendency of some commentators—and, more disturbingly, of some policymakers—to attach great significance to whether a country’s CPI score has gone up or down relative to previous years. For one thing, the sources used to construct the CPI for any given country may change from year to year—and adding or dropping an idiosyncratic source can have a substantial effect on the aggregate CPI score. For another, even when the underlying sources don’t change, we don’t know whether those sources are on the same implicit scale from year to year. And even if we put these problems to one side, a focus on changes in the final CPI score can sometimes obscure the statistical uncertainty associated with the estimated CPI—these scores can be noisy enough that changes in scores, even those that seem large, may not be statistically meaningful according to the conventional tests. Although TI always calculates statistical confidence intervals, in prior years these intervals have been buried in hard-to-find Excel spreadsheets, and the changes in CPI scores that TI highlights in its annual press releases haven’t always been statistically significant by TI’s own calculations. In an earlier post, I suggested that at the very least, TI should provide an easy-to-find, easy-to-read table assessing which changes in country scores are statistically significant at conventional levels, preferably over a 4-year period (as 1-year changes are both harder to detect if trends are gradual, and less interesting).

Apparently some folks within TI were thinking along similar lines, and I was pleased to see that in the 2017 CPI includes a reasonably prominent link to a spreadsheet showing those countries for which the 2017 CPI score showed a “statistically significant difference” from that country’s CPI score in each of five comparison years (2012, 2013, 2014, 2015, and 2016).

I’ve still got some criticisms and concerns, which—in the spirit of constructive engagement—I’ll turn to in just a moment. But before getting to that, let me pause to note my admiration for TI as an organization, and in this case its research department in particular, for constantly working to improve both the CPI itself and how it is presented and interpreted. It’s easy for folks like me to criticize—and I’ll continue to do so, in the interests of pushing for further improvements—but it’s much more challenging to absorb the raft of criticisms from so many quarters, sift through them, and invest the necessary time and resources to adapt and adjust from year to year. So, in case any folks at TI are reading this, let me first acknowledge and express my appreciation for how much work (often thankless) goes into the creation and continued improvement of this valuable tool.

Having said that, let me now proceed to raising some comments, questions, and concerns about TI’s claims about countries that appear to have experienced statistically meaningful changes in their CPI scores over the last five years. Continue reading