Living with Uncertainty

Given the problems with polls, do they accurately predict elections?  The answer is they usually do.  Their assumptions need to be more or less correct and their samples inclusive, which is harder than it has been in the past.  Nobel Prize winning physicist Richard Feynman said, “It is scientific only to say what’s more likely or less likely, and not be proving all the time what’s possible or impossible.”  

Some studies have concluded polls are no less accurate now than they have ever been, but that should not be too reassuring.  Polls have always made assumptions about who is going to vote, and wrong assumptions have long led to wrong predictions – like on how Dewey would beat Truman in 1948.  Samples were off then because people without phones supported Truman.  Samples were off in 2016 when polls included too many voter with college experience and made wrong assumptions on voter turnout. 

We never really know the outcome of an election before it happens.  At best, we know what outcome is more likely (and sometimes much more likely).  Most of us do not really have a reason to know the outcome of the election before it happens.  If we work for a partisan committee like the NRSC or DSCC, we may be concerned with allocating resources.  If we work for the media, we may feel polls are newsworthy (although I wish y’all found them less so is it really news that someone is more likely to win than someone else?). 

Polling somewhat ironically shows that voters do not trust the polls they hear about in the media – and those who took that poll may arguably have trusted polls more than the average voter or why bother.  Media coverage of bad polling can create electoral outcomes, a concern raised by a bipartisan group of pollsters (https://www.huffingtonpost.com/2010/11/08/pollsters-raise-alarm-ina_n_780705.html) back in 2010.

Modeling can help with prediction because it develops a predictive algorithm or formula that does not require a random or representative sample.  The process does require an adequate sample, however, and often more analysis and examination of error than is applied (which is the subject of a subsequent blog).  And modeling still just provides a probability and not an absolute.   Two plus two may always equal four but neither polling nor modeling are arithmetic; they say there is some level of probability that Candidate X will win, or that voter Y will support him or her.

The margin of error does not help.  It describes the statistical chance the poll is wrong by more than that number of points assuming the sample is truly random, which is rarely the case, or representative, which is increasingly arguable.

More skepticism about polls is healthy.  It reduces the risk of cutting off resources from a campaign that can win, or affecting electoral outcomes through publicizing wrong polling.  As for campaign strategy, we could use some new thinking about how we listen to voters that might make campaigns more interesting and engaging to more people, even while their outcome remains uncertain.  It is sometimes the job of campaign strategists to make what seems impossible, in fact not only possible but real, and polling alone does not do that.

Next Post:  Could AI have written this blog?