How accurate were the pre-election polls?
In the 2015 General Election, there was a systemic polling error which caused a large underestimation of the Conservative lead. In 2017, there was a wide spread of vote intentions — but, on average, underestimated Labour’s vote share by about five points.
Among the ten polling companies which conducted polling throughout the election campaign, estimates of the Conservative lead ranged from five points (Savanta ComRes) to 12 points (Kantar and Opinium). The only methodological change from prior polling was YouGov using the headline vote intention from their MRP model, instead of their traditional method.
According to my calculations based on the BBC results, the actual Conservative lead over Labour in Great Britain was 11.7 points¹.
In Great Britain, the Conservative vote share was 44.7%, and Labour had 33.0% of votes cast. Overall, the average Conservative vote intention share was 43.3% (a difference of 1.4 points) and Labour was 33.9% (a difference of 0.9 points).
In order to measure the accuracy of different polls, we are interested in:
- Lead error: the difference of Conservative and Labour vote intention share estimates minus the actual Conservative lead in Great Britain;
- Two-party mean absolute error: the average of the absolute difference between vote intention estimates and the actual vote share, for the Conservatives and Labour.
- Five-party mean absolute error: a similar calculation, but extended to the Liberal Democrats, Greens and the Brexit Party.
Of these final ten polls:
- six companies (Deltapoll, Ipsos MORI, Kantar, Opinium, Panelbase, and YouGov) estimated all five GB-wide parties correctly within two percentage points;
- seven companies (the six above plus Survation) had a two-party mean absolute error of less than 1.5 points;
- eight companies (that list plus BMG Research) correctly estimated the Conservative lead within three points;
- six companies (Deltapoll, Ipsos MORI, Kantar, Opinium, Survation, and YouGov) had a five-party error of around 1.0 points or less.
In the 2019 General Election, pre-election polls generally reflected reality: a substantial Conservative lead. Two companies — Ipsos MORI and Opinium — had an especially low five-party mean absolute error. Both of those companies estimated the rounded real results exactly for four out of the five parties.
For the three companies with somewhat larger errors (BMG Research, ICM Unlimited, Savanta ComRes), there is not a unifying methodological reason.
Whilst most vote intention polls estimate opinion for Great Britain, there is some polling aimed for Scotland, Wales and Northern Ireland. This will look at standard polling, if the final survey is conducted after 28th November.
In Scotland, the Conservatives were consistently overestimated in final polling, compared to underestimation of the SNP vote share.
In Wales, YouGov’s final poll was notably accurate: with the estimate for the two largest parties out by a single percentage point.
In Northern Ireland, LucidTalk hit two of the three smaller parties (APNI and UUP) exactly, with some error on the DUP and SF shares.
Surveys are incredibly important, as censuses are usually infeasible. Polls offer an imperfect view — a hazy image of current opinion, and a dusty mirror on society. In this election, vote intention polling was broadly accurate.