Journal of Irreproducible Results

There really is a spoof journal with this title, and very funny it is, too -- check it out here.

I'm thinking, though, that a couple of distinguished economists might not find it quite so funny at the moment. Back in 2010, Kenneth Rogoff and Carmen Reinhart published an article, under the auspices of the American Economic Association (AEA) titled "Growth in a time of debt".  They claimed to demonstrate that any time a country's debt topped 90% of GDP, its growth rate was liable to suffer -- averaging about 1% per annum lower than for less indebted countries.

You couldn't claim that this led to the craze for austerity economics -- that had begun much earlier -- but it certainly chimed with the zeitgeist, and provided academic reinforcement for the likes of George Osborne in the UK and the GOP caucus in the United States.

Fast forward now to the present, and Rogoff/Reinhart is at the centre of one of the juiciest academic scandals in some time. A new paper by Herndon, Ash and Pollin calls the paper's entire thesis into question.  This later article came into being when Herndon tried and failed to reproduce the results in the earlier piece.  I can't do better than to quote from the abstract on the PERI website:

Herndon, Ash and Pollin replicate Reinhart and Rogoff and find that coding errors, selective exclusion of available data, and unconventional weighting of summary statistics lead to serious errors that inaccurately represent the relationship between public debt and GDP growth among 20 advanced economies in the post-war period. They find that when properly calculated, the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0:1 percent as published in Reinhart and Rogo ff. That is, contrary to RR, average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.

Wow!  Just about every available academic sin in one article!  Rogoff/Reinhart have, of course, defended themselves, admitting to coding errors but arguing that even if all of the data are included, their conclusions are still largely correct.  Given the nature of the "coding errors" -- one of them was the inclusion of only one year of data for New Zealand, 1951, a year in which GDP fell by more than 7% -- my own first instinct was to be skeptical of their excuse.  On further thought, however,  I'm inclined to believe them, mainly because they willingly handed over their data set to the Herndon team, something they would hardly have done if they knew it was bogus.

Still, you have to wonder how two distinguished academics let such a flawed piece see the light of day.  It seems more than likely that Rogoff and Herndon's "Bayesian prior" in this case was a belief that excessive debt caused slow growth.  That's what they expected to find, and when that's what came out of the data, they didn't check that data as closely as they should have.  That's not good enough, especially when the conclusions they reached reinforced damaging economic policies in so many parts of the world.

At least Rogoff and Reinhart didn't destroy their data after they had processed it, unlike the notorious climate change team at the University of East Anglia in Britain, who "proved" anthropogenic climate change to their own satisfaction, and then destroyed reams of historical data.  Their results, which continue to inform public policy on the issue in spite of the highly suspect circumstances, are quite literally irreproducible.  Amazingly, this sort of academic finagling is not uncommon in the climate change community -- see, for example, this evisceration of the so-called "hockey stick" temperature graph.

Do Rogoff and Reinhart have anything in common with the warmists?  Both seem keen to draw conclusions that they think people want to hear, and are maybe not as careful as they should be about how they get there.  Maybe this has something to do with a remarkable little piece in the NYT this week, which noted that there are now about 4,000 quasi-academic journals on the internet, soliciting articles from anxious authors and then charging them for the privilege of seeing their masterpieces in print.  We may be seeing the operation of a new version of Gresham's Law, with bad science driving out good.