The Science of Error: how polling botched the 2016 election (Synopsis) [Starts With A Bang]

“Distinguishing the signal from the noise requires both scientific knowledge and self-knowledge.” -Nate Silver

When you take a poll, you survey a number of people with an opinion about something in an attempt to predict the behavior of a much larger number of people. If you increase the number of people you poll, your poll uncertainty drops. This reduction in what we call a statistical error will mean your polls reflect the likely outcome better and better, given one assumption. You have to assume that data obtained from the people you’re polling are reflective of a random sample of future voters.

A visualization of how your statistical uncertainty drops as your sample size increases. Image credit: Fadethree at English Wikipedia.

And that’s a big assumption! Any deviation from that, in turnout, in voter preference, in sampling bias, etc., will mean that there are additional sources of error that you have no way of accounting for. These systematic errors plague all observational and measurement sciences, and predicting an election’s outcome is no exception.

Truman holding up a copy of the infamous Chicago Daily Tribune after the 1948 election. Image credit: flickr user A Meyers 91 of the Frank Cancellare original, via https://www.flickr.com/photos/85635025@N04/12894913705 under cc-by-2.0.

The successes of predictive models in determining the outcome in 2012 gave us an unwarranted confidence in 2016, which should serve as a rude awakening for us all.

from ScienceBlogs http://scienceblogs.com/startswithabang/2016/11/09/35381/

“Distinguishing the signal from the noise requires both scientific knowledge and self-knowledge.” -Nate Silver

When you take a poll, you survey a number of people with an opinion about something in an attempt to predict the behavior of a much larger number of people. If you increase the number of people you poll, your poll uncertainty drops. This reduction in what we call a statistical error will mean your polls reflect the likely outcome better and better, given one assumption. You have to assume that data obtained from the people you’re polling are reflective of a random sample of future voters.

A visualization of how your statistical uncertainty drops as your sample size increases. Image credit: Fadethree at English Wikipedia.

And that’s a big assumption! Any deviation from that, in turnout, in voter preference, in sampling bias, etc., will mean that there are additional sources of error that you have no way of accounting for. These systematic errors plague all observational and measurement sciences, and predicting an election’s outcome is no exception.

Truman holding up a copy of the infamous Chicago Daily Tribune after the 1948 election. Image credit: flickr user A Meyers 91 of the Frank Cancellare original, via https://www.flickr.com/photos/85635025@N04/12894913705 under cc-by-2.0.

The successes of predictive models in determining the outcome in 2012 gave us an unwarranted confidence in 2016, which should serve as a rude awakening for us all.

from ScienceBlogs http://scienceblogs.com/startswithabang/2016/11/09/35381/