Case Study: The Reinhart-Rogoff Error — When a Spreadsheet Shapes a Continent

The Paper

In January 2010, Harvard economists Carmen Reinhart and Kenneth Rogoff published a paper titled "Growth in a Time of Debt." The paper's central finding: countries with government debt exceeding 90% of GDP experienced sharply lower median economic growth — a decline from roughly 3% growth to -0.1%.

The finding was instantly influential. It provided a specific, quantifiable threshold — exactly what policymakers needed. European Commission Vice President Olli Rehn cited the 90% threshold explicitly as justification for austerity measures. Paul Ryan, then chair of the US House Budget Committee, cited it in budget proposals. The paper became, as economist Paul Krugman later put it, "the most influential economic paper of the early post-crisis period."

The Error

In April 2013, Thomas Herndon, a graduate student at the University of Massachusetts Amherst, attempted to replicate Reinhart and Rogoff's results for a class assignment. He couldn't reproduce them. After obtaining the original spreadsheet from the authors, he discovered three problems:

  1. A coding error in Excel. Reinhart and Rogoff had accidentally excluded five countries from their averaging formula. The countries (Australia, Austria, Belgium, Canada, and Denmark) were in the spreadsheet but were not captured by the cell range in the formula.

  2. Selective exclusion of available data. Data for several countries in the early post-war period were excluded without clear justification.

  3. Unconventional weighting. The averaging method gave equal weight to each country regardless of the number of years it spent in the high-debt category. New Zealand's single year of high debt and -7.6% growth received the same weight as the UK's nearly two decades of high debt with positive growth.

When the coding error was fixed and the excluded data included, the sharp threshold at 90% largely disappeared. Countries with debt above 90% of GDP still showed somewhat lower growth — but the relationship was modest, not dramatic, and the specific 90% threshold was an artifact of the errors.

The Policy Consequences

The Reinhart-Rogoff paper was published in January 2010. The errors were discovered in April 2013. During those three years:

  • The European Commission used the 90% threshold as intellectual justification for austerity requirements imposed on crisis-hit countries.
  • Greece, Spain, Portugal, Ireland, and other countries implemented deep spending cuts partly justified by the Reinhart-Rogoff framework.
  • Youth unemployment in Greece exceeded 60%. Youth unemployment in Spain exceeded 55%.
  • The UK implemented austerity measures under Chancellor George Osborne, who cited the 90% threshold.
  • Multiple countries experienced double-dip recessions that many economists attributed partly to premature austerity.

Whether the austerity policies were entirely a consequence of the Reinhart-Rogoff paper is debatable — political preferences for fiscal conservatism predated the paper. But the paper provided the specific, quantifiable intellectual foundation that made austerity policies defensible in policy debates. Without the 90% threshold, the argument for austerity was weaker.

Failure Mode Analysis

Precision Without Accuracy (Ch.12)

The 90% threshold was a specific, precise number — and that specificity was its political power. "Countries above 90% debt-to-GDP experience negative growth" is actionable. "There may be some relationship between debt and growth, depending on context, institutions, and macroeconomic conditions" is not. The precision was false, but it was exactly what the policy environment demanded.

Replication Failure (Ch.10)

The error went undetected for over three years because the original data and code were not shared. When a graduate student attempted replication — the most basic quality check in science — the error was found immediately. If the data had been publicly available from the start, the error would likely have been caught within weeks.

Authority Cascade (Ch.2)

Reinhart and Rogoff were among the most prominent economists in the world. Their institutional affiliation (Harvard) and their previous work (This Time Is Different, a widely praised book on financial crises) gave their findings maximum credibility. Policymakers cited the authority of the authors rather than independently evaluating the evidence.

Plausible Story Problem (Ch.6)

"Too much debt causes economic decline" is a narrative that aligns with common-sense intuitions about debt (household analogy: families that borrow too much get into trouble). The finding confirmed what many policymakers already believed, making it resistant to critical examination.

The Aftermath

Reinhart and Rogoff acknowledged the coding error but argued that their broader point — that very high debt is associated with lower growth — remained valid even after corrections. This is technically true but misses the point: the specific 90% threshold, which was the policy-relevant finding, was an artifact of errors. The corrected finding ("there's some relationship between debt and growth, but it's complicated") was not the finding that shaped policy.

The episode did not change economics' institutional practices in any fundamental way. Data sharing remains voluntary in most economics journals. Replication is still disincentivized. High-profile papers by prestigious economists still receive less scrutiny than they should.

Analysis Questions

1. The error was discovered by a graduate student doing a class assignment — not by peer review, not by the economics profession's quality control mechanisms. What does this tell us about the profession's correction infrastructure?

2. Apply the five properties of paradigm-breaking crises (Chapter 19) to the Reinhart-Rogoff revelation. Was it visible enough, undeniable enough, costly enough, attributable enough, and repeated enough to force institutional change?

3. The coding error was in Excel — a software tool that does not track changes, does not flag potential formula errors, and does not support the kind of reproducibility that statistical programming languages (R, Python, Stata) enable. Should economics journals require the use of reproducible computing environments? What resistance would this face?

4. Compare this case to the opioid crisis (Chapter 23, case study 1). Both involved influential claims that were based on inadequate evidence and that caused significant harm. Why did the Reinhart-Rogoff error produce a faster (though still limited) correction than the opioid crisis?

5. Design an institutional mechanism that would have caught the Reinhart-Rogoff error before it influenced policy. Be specific about what would be required, who would do it, and how it would be funded.

Key Takeaway

The Reinhart-Rogoff error is a nearly perfect case study in how academic economics translates into policy harm. Every failure mode in the chain is identifiable: a prestigious claim (authority cascade) with a specific number (precision without accuracy) that confirmed existing beliefs (plausible story) was accepted without verification (replication failure) and used to justify policies with enormous human consequences. The error was found by the most basic act of scientific hygiene — a replication attempt — that the profession's institutional structures had failed to perform for three years.