Soon there will be a plethora of “horse race” polls in various races and nationally, likely showing divergent results. That is a seasonal phenomenon plus the one-two punch of the Supreme Court decision on Roe v. Wade and the compelling hearings by the House Select Committee on the January 6 Attack have made the mid-terms more interesting and more contested. Plus, Republicans have nominated some truly dreadful candidates.
On the flip side, President Biden’s approval is low and the economy is perceived as weak. Last time there was a mid-term election with rising inflation (although rising less than now at that point) and a then unpopular Democratic President, the year was 1978. In that year, which almost no one under age 50 remembers, Republicans picked up three Senate seats and 15 House seats.
There are many differences between 1978 and 2022. So this year is looking interesting, but we cannot know what will happen based on polling alone, or even mostly. Here are two caveats on polling and then some things to watch for since polling isn’t going away.
The first caveat should be familiar to anyone who has followed this blog: No one is polling random samples. Polling simply doesn’t work the way it used to work. The rule for a random sample is that everyone in the population of interest – people who will vote on November 8, 2022 – has an equal chance of being included in the poll. Caller ID means no one has been able to do that since about 1990. Instead, pollsters mimic the electorate and aim for representative samples. Widely swinging turnout, extreme difficulties reaching some demographics, and response bias have made “representativeness” more and more difficult. Response bias is not unitary. It combines distrust of polls, distrust of the media, the desire to be seen as an individual not part of a labeled aggregate, the quality of the polling questions (which may cause terminations) and varying interest in politics. The response bias factor is greater among conservative voters than liberal voters but is greatest of all in the middle of the ideological spectrum, which is often the most interesting. Pollsters correct for the problems by “weighting” the data. Those weights rely on assumptions about the shape of electorate, predicated in part by mistakes of the past. There is no correction for “new” errors until after they have happened.
No poll, no matter how carefully constructed, should ever be seen as an absolute; an indication, yes, but not an absolute. If polls – even multiple polls – show Candidate A two points ahead of Candidate B, that does not mean Candidate A will win. That has nothing to do with the margin of error (which is addressed by multiple polls). It has more to do with common assumptions about turnout and perhaps new forms of response bias which we don’t know about yet. (On the other hand, if even two even semi-legitimate polls show Candidate A winning by 15 points, he or she very likely will win handily; this is all on the margins.)
Which brings me to my second caveat: Change is more often slow and incremental than sudden and dramatic. Small things happening slowly are worth attending to. Polls do not often pick these up, but deeper conversations with voters can at least create hypotheses ahead of the polls. I have found stories about how most Republicans still support Trump uninteresting. Of course they do; it is more important that the support appears less than it was. On the flip side, there is no question that many Democratic and independent women are deeply upset by the Roe v. Wade decision. The question is whether some of those who would not ordinarily vote in the midterms will turn out and vote as a consequence. Younger, lower propensity voters are not often included in polls, or not included in numbers that would allow analysis of them separately from the aggregate. In a close election, however, they could make a critical difference.
So, if you want to know what will happen, be skeptical but not dismissive of polls, be analytic about what small(ish) things may matter, and be curious about probing whether they will this time. Big upsets don’t come out of nowhere. They are also rare. But they do happen. Here are some things to watch for in polls and in the world outside them.
Small groups and sample sizes: Surprises may be produced by small groups in the electorate, like independent women, young voters, high propensity voters who stay home, low propensity voters who turn out. But an aggregate poll, even with a large sample size, cannot tell you much about groups who may be 10 percent of the electorate.
For example, the poll by Cygnal of the Georgia electorate led off the silly season. This Republican outfit put out a release on their 1200 sample Georgia poll, which included an oversample of 770 African American voters. First, they trumpeted that Republican Governor Brian Kemp led Democrat Stacey Abrams by 50 to 45 percent, which is pretty unimpressive for an incumbent Governor, actually, and they never defined their turnout assumptions which will matter a great deal in this race. Next, they declare that African American voters are not homogeneous, which really didn’t require a poll, and then declare that a quarter of African Americans under age 35 are supporting Kemp. Valid? Maybe, but maybe not. Even with a sample of 770 African American voters, how many were under age 35? I would guess maybe 150, and maybe fewer actually and upweighted to about 150. Was that small sample representative of younger African Americans? And who was the base sample of anyway, those who have voted previously or all African Americans? It may be true that Kemp is doing better than expected among younger African Americans – he has gotten a lot of press lately for not being as opposed to voting rights as other Republicans – but the conclusion, without adequate information about the sample, that this is somehow a prediction, seems a bit dubious.
Turnout assumptions: I have seen very few public polls (actually I cannot recall one) that have specified assumptions about voter turnout. First, if turnout is based on self-report (reaching people at random and screening for whether they are likely to vote), the poll is likely representing turnout as both higher than it will be (there is social pressure to say likely to vote) and more Democratic (see above on response bias). If they are selecting people based on vote history from a voter file, well, fine, but then say so, and even then is the assumption that turnout patterns are like those of 2018 (which was a Democratic year) or different. In any case, if the information on weighting and turnout is absent, the poll is really hard to read in a meaningful way.
Defining independents: With deeper party polarization, looking at voters who say they are independent is an important task. The thing is, who they are is often volatile and independents are often undersampled in polls. The image is that independents are people without party predilections, but there are independents who call themselves that because neither party is sufficiently liberal or conservative enough for them. They may be independent, but they are not up for grabs. Then there are those who don’t like either party because they feel neither party represents their moderate or centrist views. They are very important, but their distaste for politics extends to a distaste for polls, and so they are often undersampled even while many of them vote. People also float in and out of calling themselves independents, depending how they are feeling about the parties which creates exaggerated volatility – unhappy Republicans may say they are independent, making the category more Republican, or vice versa. Mushing all this together independents are still generally only about 20 to 25 percent of the polling sample. See the above on sample size. A poll of 500 with roughly 100 “independents,” of multiple descriptions won’t say much about them. (Except in some places, like California, where DTS – decline to state – voters can be a constant category.)
Extrapolating from national data: There are new national polls that show the generic match up between Democrats and Republicans far closer than it was in the spring even while President Biden’s numbers are low. It’s intriguing, and hints that the midterms may be less of a victory for the Republicans than previously thought. Hints at. Intriguing. I’m on board with both of those but not (yet) with any prediction. There is no reasonable way to extrapolate the national data into a number of house seats or to any particular statewide race. First, the national data may simply represent an even greater separation between “red” and “blue” places than was true even six months ago. Most of the national polls do not provide a time series by region, plus the above comment on turnout applies. National data often inadvertently oversample the coasts because there are more phones per person on the coasts. That New York, New England, and California are approaching political apoplexy matters, but doesn’t predict what voters will do in Georgia, Ohio, Wisconsin, or Nevada.
So what to do with all this if you are interested in the election? Well, that depends on your vantage point.
If you are a candidate, go figure out how many votes you need to win the election, how many you have already effectively banked by virtue of your party label or prior base, who are the additional people by demographics, geography, and perspective you need to win, and then go plan your campaign to win them. Whether you are ahead or behind, and by a little or a lot really doesn’t matter much in doing the intellectual work of how to win. Polling can help you understand your district better, and what people there may want to know about you and your opponent, but the strategic process of what you do with that information matters a lot more than the poll per se, which is only one tool in developing your strategy.
If you are a member of the press, use the polls to guide who you talk to, what you assess, and to enlarge your view of the range of what might happen. I wish you wouldn’t report on them as much but I recognize that is a losing battle. It’s too easy to report polls. But please ponder the questions that emerge from them: Will Republican voters turnout in droves because they are upset with Biden, or will more than usual stay home because they have new doubts about the MAGA crowd? Is there something happening that will cause younger people and lower propensity Democrats to turn out, whether that is something local or national? Are the individual candidates and campaigns perceived as interesting or distinctive enough (in ways both positive or negative) to break through whatever is happening nationally? And do voters generally believe they have relevant choices in the district or state on which you are reporting, or are the candidates boring, muting any opportunities for changing turnout dynamics or partisanship?
If you are an activist, well, go get to work. Whether your candidate is ahead or behind, by a little or a lot, door-to-door canvassing to discuss the election matters and has more of a lasting impact than most anything else, even for the next election. Read the polls if you like. But don’t let predictions become a self-fulfilling prophecy, as too often happens during the silly season.
ViewfromthePearl.com 