Case Study 20.1: The 2015 UK General Election — Britain's Polling Disaster

Background

The United Kingdom's May 7, 2015, general election was supposed to produce a hung parliament. Every major British polling firm had been tracking the race for months, and for much of the campaign, the Conservatives under David Cameron and Labour under Ed Miliband were essentially tied — typically within one or two points of each other, with no party anywhere near the 326 seats needed for an outright majority in the 650-seat House of Commons.

The polling consensus was so strong and so consistent that political commentators, financial markets, and bookmakers had all priced in a hung parliament as the near-certain outcome. Scenarios involving Conservative-Liberal Democrat or Labour-Scottish National Party coalitions were extensively analyzed. The question was not whether there would be a hung parliament but who would be the senior partner.

The actual result, when polls closed at 10 p.m. and the exit poll was announced, was a shock to nearly everyone in British politics: the Conservatives won 331 seats — a clear outright majority — while Labour won only 232. The vote share gap was also substantial: Conservatives at 36.9 percent to Labour's 30.4 percent. In competitive constituencies across England, the Conservatives outperformed the polling average by 4 to 6 points.

What the Polls Said

In the final week of the campaign, the polling average showed: - Conservatives: 33–34 percent - Labour: 33–34 percent

The actual result was Conservatives 37 percent, Labour 30 percent — a swing of approximately 7 points from the polling average to the actual result.

This is the largest systematic polling error in modern British electoral history. No individual poll in the final weeks came close to predicting the outcome. The range of final-week polls showed a maximum Conservative lead of 4 points (one poll) against a minimum of -3 (Labour leading by 3). The actual Conservative lead was approximately 6.5 points.

The Inquiry

Following the election, the British Polling Council and the Market Research Society jointly commissioned an independent inquiry led by Patrick Sturgis, a professor of quantitative social science at the University of Southampton. The inquiry report, published in March 2016, reached several significant conclusions.

Finding 1: Unrepresentative Samples

The primary cause of the polling failure was that polling samples were unrepresentative of the actual electorate. Specifically, Labour supporters were overrepresented in polling samples and Conservative supporters were underrepresented, not because of overt partisan bias but because of the demographic composition of who actually responds to polls.

As in the United States, the British polling industry had shifted substantially from random-digit telephone dialing (which achieves something closer to probability sampling) to online panels of recruited volunteers. These panels allowed cost-effective, rapid polling but introduced systematic compositional biases. The people who volunteer for online polling panels are not a random cross-section of the population; they are disproportionately engaged with politics, have stronger partisan identities, and — in the British context of 2015 — appear to have been disproportionately left-of-center in their politics.

Finding 2: The "Shy Tory" Debate

A second hypothesis received substantial attention: the "Shy Conservative" effect, in which potential Conservative voters were reluctant to identify themselves as such to pollsters due to social stigma. This theory had been advanced after previous Conservative polling underestimates (notably 1992, when polls also significantly overestimated Labour).

The Sturgis inquiry concluded that the evidence for Shy Toryism in 2015 was weaker than commonly believed. The shy Tory effect requires that voters accurately know their own preferences but deliberately conceal them from pollsters — a different mechanism from simply being less likely to participate in polls at all. The inquiry found more evidence for the second mechanism (differential participation) than the first (deliberate concealment), though it acknowledged that both might have been present.

Finding 3: Herding

The inquiry also documented strong evidence of herding. The distribution of final-week poll results was statistically too tight — showing much less variance across polling houses than would be expected if polls were independently measuring the same underlying quantity. Several polling firms interviewed for the inquiry acknowledged that they monitored competitors' results and that significant outliers prompted internal review of methodology that could result in adjustment.

The practical consequence: the polling consensus masked the genuine uncertainty in the underlying data. If individual firms had published their raw results without adjustment toward the consensus, the distribution of final polls would have shown a wider range of outcomes — including some that accurately predicted the Conservative majority.

Aftermath and Industry Response

Following the inquiry, the British Polling Council adopted a set of recommended methodological standards: - Mandatory public disclosure of detailed methodology, including weighting variables and sample composition - Recommendation against adjusting results toward industry consensus - Encouragement of experimental polling designs that could test for differential nonresponse

Several major British polling firms invested substantially in methodological reform. YouGov, in particular, developed its Multilevel Regression and Poststratification (MRP) model for constituency-level estimates — a technique that would perform well in the 2017 snap election, accurately predicting the hung parliament that the simpler polling average missed.

Discussion Questions

1. The inquiry found that sample unrepresentativeness (not "shy Toryism") was the primary cause of the 2015 miss. What methodological implications follow from this distinction? Would the fix for a shy Tory problem be different from the fix for a differential participation problem?

2. Compare the incentive structure that produced herding in the UK polling industry to the incentive structure that produces it in the United States. Are the structural features the same? Are there features of the British market (fewer but larger polling firms, a more concentrated media landscape) that might make herding more or less severe than in the U.S.?

3. YouGov's MRP model performed well in 2017 after performing poorly in 2015 with standard methods. What does this suggest about the relationship between methodology innovation and specific political contexts? Is it possible that MRP solved the 2015 problem but would not solve a different structural problem in a future election?

4. The 2015 British polling failure and the 2016 American polling failure share structural similarities but occurred in different political systems (Westminster parliamentary vs. presidential). What aspects of the failure seem to be system-agnostic (present in both contexts) and what aspects might be system-specific?

Analytical Exercise

Using the following hypothetical data, assess whether herding is occurring:

Calculate the mean and standard deviation of the polling estimates. Under a simple sampling error model with n=1,000 per poll, what would you expect the standard deviation of poll results to be? What does the comparison suggest about whether herding occurred? How does the systematic direction of the miss (all polls underestimated Conservatives) relate to the herding question?