The Economist’s recent cover story, introducing what it calls the “Crony-Capitalism Index”, has generated a lot of buzz. The study ranks 23 countries (counting Hong Kong separately) based on the Economist’s calculation of the prevalence of politically connected business dealing. The study takes billionaires from the Forbes Billionaires List who are primarily active in certain industries (such as casinos, banking, extractive industries, real estate, utilities, etc.) that the Economist deems “rent-heavy,” and looks at these billionaires’ share of the economic pie in their country. The index has already been used as the basis for media criticism of those countries that scored poorly, such as Hong Kong (1st) and Malaysia (3rd) — indeed, the Malaysian government was so upset that it censored the Economist for the week the index came out.

Some of the results are unsurprising: Russia and India score fairly high in this measure of crony capitalism, whereas Germany bottoms out the list. But other results are more puzzling.  Not only does the index report that Hong Kong has more crony capitalism than mainland China, but also that mainland China has less crony capitalism than either the United States or Great Britain. What gives? Does the United States really have more of a crony capitalism problem than China?

Maybe not. The index suffers from a host of limitations, a few of which the Economist acknowledges, that may explain why some of the results seem so strange. In fact, the results may say more about the methodology of the study and the challenges of comparative measures of corruption than they do about the countries being assessed. Indeed, it’s not clear that this crony-capitalism index tells us anything useful about the quality of institutions or the competitiveness of markets in the countries it ranks.

The crony-capitalism index may pick up the extent of crony capitalism in each country ranked, but the ranking is also likely to be influenced by — and therefore skewed by — a range of factors that have little to with the prevalence of crony capitalism per se:

• First, the index is likely to be biased against countries in which a large share of the economy is in the Economist‘s “rent-heavy industries,” regardless of how much rent-seeking is actually going on in those industries. For example, in states like Russia (or, for that matter, Norway, which is not included in the index), the natural resource sector is not only large, but it makes up an outsized portion of the economy when compared to, for example, the United States. Likewise, in small city-states like Hong Kong and Singapore, there’s a lot of money to be made in real estate (much as there is in New York City or London), relative to the overall economy of those relatively small jurisdictions. Moreover, as the Economist candidly acknowledges, the index presumes that an industry is as rent-heavy in one country as another. But there’s no reason to assume that oil exploration rights are handed out the same way in Great Britain as in Russia, or that electrical utilities are as able to capitalize on political connections in the United States as in Indonesia. Thus the index is biased against countries dominated by the industries identified as “rent-heavy,” independently of the amount of actual rent-seeking behavior going on in those industries.
• Second, the Economist study only looks at billionaires (a point it acknowledges). Although the Economist acknowledges this limitation and says it means its index is only a “rough guide to the concentration of wealth in opaque industries,” the bigger problem — which the Economist does not acknowledge explicitly — is that using this cutoff may bias the results so that larger economies, or richer countries, score worse. After all, in smaller economies there may be many oligarchs whose wealth runs merely into the hundreds of millions of dollars, rather than billions — a phenomenon that Forbes recently noted in Ukraine. As a result, smaller economies may be underrepresented in the billionaires list — even if the amount of cronyism is equal or greater — biasing the index against richer countries with larger economies. For similar reasons, the methodology for constructing the index is likely to bias the results against those countries that have privatized these rent-heavy sectors. For example, Russia’s privatization of state-owned industries has created a set of oligarchs with billions in wealth. But in China, as the study notes, many allegedly rent-heavy industries remain dominated by government-owned businesses. These state-owned enterprises may receive similar or greater privileges than privatized companies, but their ownership structure may not result in the creation of billionaires. Why should a state-owned enterprise not count as the beneficiary of “crony capitalism”, just as much as a fully privatized company?
• Third, again as the Economist also acknowledges, the wealth of crony capitalists is sometimes hidden or dispersed. In China, the wealth of Bo Xilai’s family, which The New York Times reported came largely through links to the state, has been concealed and dispersed among several family members. Or consider a man who has been in the news a lot lately — Vladimir V. Putin. He’s nowhere to be found on the Forbes list, but some estimates have listed him as the wealthiest man in the world based on secret wealth allegedly obtained through his domination of the Russian political system. Again, the issue is not merely inaccuracy (which the Economist acknowledges), but bias: in those countries where it’s either easier to hide one’s wealth, or where there’s a stronger incentive to do so, the crony-capitalism index will understate the true amount of cronyism and rent-seeking going on.
• Fourth, there’s the focus on personal wealth itself, as opposed to corporate wealth. The Economist notes that America does relatively well in the study because unlike other countries its “energy billionaires” barely register in the overall economy. But that is partly an accident of history: The U.S. energy industry was dominated by extremely wealthy individuals for much of its early development. While those individuals have left the scene and their wealth has become fairly dispersed, many of the public companies those early billionaires created remain strong incumbents and continue to receive many kinds of government support. Likewise, the Economist notes that U.S. financial services billionaires are also fairly small in number and wealth compared to, say, high tech billionaires, making the individual contribution to this index of the U.S. financial services industry fairly small. But to Americans who have just witnessed six years of multi-hundred-billion-dollar financial industry bailouts overseen largely by alumni of the financial industry, the argument that the industry as a whole doesn’t extract enormous rents seems to depend entirely on a focus on personal rather than corporate wealth — on the crony, not the crony company.

The bottom line is that although lists and rankings are always fun and tend to generate buzz, this so-called “crony-capitalism index” probably doesn’t tell us much about relative rates of crony capitalism in these countries. Perhaps rather than using some easily accessible data like the number of billionaires, it might be more fruitful to look at data that measure the actual relationships between these companies and their governments. For example, levels of state support through government-backed loans or subsidies, or the number and kind of political connections between successful firms and governments.