Problems with polling: Redux

I haven’t posted since April since I had little to contribute to what I saw as the two overarching goals of the last six months: electing Joe Biden and developing a COVID vaccine. I did my civic duty toward the first and had nothing to contribute to the second, and so it seemed a time to pause. Now, I feel my free speech is restored and for a moment at least there is some attention to one of my favorite topics – how we need to do research for campaigns differently. I have covered much of that previously but here is a redux on the problems with polling with some updating for 2020.

1. Samples are not random. If you ever took an intro stats course, it grounded most statistics in the need for a random sample. That means that everyone in the population of interest (e.g. people who voted November 3) has an equal probability of being included in the sample.

The margin of error presumes a random sample. The number of people required to give you an accurate picture of the array of views in a population depends on size of the sample, the breadth of the views in the population, and the randomness of the sample.

The intuitive example: Imagine a bowl of minestrone soup. If you take a small spoonful, you may miss the kidney beans. The larger the spoonful (or sample) the more likely you are to taste all the ingredients. The size of the spoon is important but not the size of the bowl. But if you are tasting cream of tomato soup, you know how it tastes with a smaller spoon. America is definitely more like minestrone than cream of tomato.

The problem with polling has little to do with the margin of error, which remains unchanged. The problem is that pollsters have not used random samples for a generation. The advent of caller ID and people’s annoying proclivity to decline to answer calls from unknown numbers (a proclivity I share), plus some changes in phone technology with fiber optics – including a proliferation of numbers that are not geographically grounded, and an explosion of polls and surveys (How was your last stay at a Hilton?), makes the act of sharing your opinion pretty unspecial.

Not to worry, we pollsters said. Samples can still be representative.

2. The problem with “representative” samples. A representative sample is one constructed to meet the demographics and partisanship of the population of interest (e.g. voters in a state) in order to measure the attitudes of that representative sample.

The researcher “corrects” the data through a variety of techniques, principally stratified samples and weighting. A stratified sample separates out particular groups and samples them separately. Examples include cluster samples, which stratify by geography, and age stratified samples, which use a separate sample for young people, who are hard to reach.

Professional pollsters usually sample from “modeled” files that tell how many likely voters are in each group and their likely partisanship. They upweight – or count the people they are short of extra. They may up-weight the conservative voters without college experience, for example, to keep both demographics and partisanship in line with the model for that state or population. Virtually every poll you see has weighted the data to presumptions of demographics and partisanship.

Back to the minestrone soup example: Samples are drawn and weighted according to the recipe developed before the poll is conducted. We presume the soup has a set quantity of kidney beans because that’s what the recipe says. But voters don’t follow the recipe – they add all kinds of spices on their own. Pollsters also get in a rut on who will vote – failing to stir the soup before tasting it.

Most of the time, though, the assumptions are right. The likely voters vote and the unlikely voters do not, and partisanship reflects the modeling done the year before. But disruptive events happen. In 1998 in Minnesota, most polls (including my own) were wrong because unlikely voters participated and turnout was unexpectedly high particularly in Anoka County, home of Jesse Ventura, who became Governor that year. That phenomenon is parallel to the Trump factor in 2016 and even more so in 2020. Unexpected people voted in unexpected numbers. If the polls are right in 2022, as they generally were in 2018, it is not because the problem is fixed but because conventional wisdom is right again, which would be a relief to more than pollsters, I expect.

3. What’s next. I hope part of what’s next is a different approach to research. If campaigns and their allies break down the core questions they want to answer, they will discover that there is a far bigger and more varied toolbox of research techniques available to them. The press could also find more interesting things to write about that help elucidate attitudes rather than predict behavior.

Analytics has a great deal more to offer. That is especially so if analytics practitioners became more interested in possibilities rather than merely assigning probabilities. Analytics has become too much like polling in resting on assumptions. Practitioners have shrunk their samples and traded in classical statistics for solely Bayesian models.

Please bear with me for another few sentences on that: classical statistics make fewer assumptions; Bayesian statistics measure against assumptions. When I was in grad school (back when Jimmy Carter was President – a Democrat from Georgia!), people made fun of Bayesian models saying it was like looking for a horse, finding a donkey, and concluding it was a mule. We will never collect or analyze data the way we did in the 1970s and 80s, but some things do come around again.

It would also be helpful if institutional players were less wedded to spread sheets that lined up races by the simple probability of winning and instead helped look for the unexpected threats and opportunities. In those years when everything is as expected, there are fewer of those. But upset wins are always constructed by what is different, special, unusual, and unexpected in the context of candidates and moment. Frankly, finding those is what always interested me most because that’s where change comes from.

More on all of this in the weeks and months ahead, and more on all the less wonky things I plan to think about Democrats, the south, shifting party alignments, economic messaging, and my new home state of Mississippi. I am glad to be writing again, now that I feel more matters in this world than just Joe Biden and vaccines.

My Vote on March 10 for…

I had always planned to wait until the March 10 Mississippi primary in which I will vote to decide for whom I would vote. But I have now decided – and have surprised myself (and will surprise many who know me). I considered many factors, superficial and otherwise, before deciding that I will vote for Joe Biden.

I considered relationships with campaign operatives and even with some candidates. After spending 35 plus years in Democratic politics, I unsurprisingly know people in almost every campaign. I am probably closer to more people in Senator Klobuchar’s campaign than any other because of shared history in Minnesota politics, and I know Senator Klobuchar slightly and like and respect her. But that is not enough.

I also know slightly and very much like and respect former Mayor Buttigieg, who (with some gaps) has run a remarkable campaign.  For a short while, the former Mayor of South Bend was the delegate leader and is still solidly in the top three in delegates.  He has tremendous talent and smarts, a message of meaning and substance, and we do, indeed, need to bring a new generation of leadership into the Democratic Party.  But that is not enough.

If I could wave a magic wand and make one of the candidates president, it would be Senator Elizabeth Warren.  Her politics are closest to my own, she cares about the things I care about (and she even did a town hall in Mississippi).  I love her energy and I do so very much want to see a woman President in my lifetime.  My 35 plus years in politics only makes that drive stronger as politics is not a field where women are near equal to men in how we are treated or heard.  That should be enough reason to support Senator Warren but this year it is not.

Because this isn’t about me.  It is about our country, our system of democracy, and about people whose very lives (unlike my own) depend on the outcome in November. 

I am not hostile to Bernie Sanders, whose political ideas may be closer to mine than Joe Biden’s are.  I don’t blame him for Hillary Clinton’s loss to Donald Trump, which was far more complicated than that.  I will support him with enthusiasm if he is the nominee.  And I believe there is a chance he might eek out a victory if by November Donald Trump is less popular than he is right now.  But there is also a significant risk that he would lose and that he take down good people running for the Senate and the House with him.  That is too big a risk.

Then there is Michael Bloomberg.  I believe he is trying to do good in this race and in large part because he saw a gap that, before South Carolina, worried me as well.  But I believe he has the wrong positioning and politics for the task at hand, which is to defeat Donald Trump.

I support my candidate, Joe Biden, because it is, I believe, the right thing to do and that he is the right candidate for this time. It is not personal.  I have never met Joe Biden – we don’t share the same political space.  It may not be what I want but it is what we need:  A good and decent man, who listens to others, who wants to unite the country around shared ideals of democracy. 

He has my vote on March 10.

Big Structural Change

Increasingly, the Democratic presidential nomination seems a battle between former Vice President Joe Biden and Senator Elizabeth Warren.  Senator Bernie Sanders impacts the race but with scant signs of growth in his support.  Senator Kamala Harris and Mayor Pete Buttigieg still hold on to smaller constituencies, with life in other candidacies, including Senators Booker and Klobuchar, and with flashes of passion from former Congressman O’Rourke. 

There is still time for another candidate to emerge but the race has remained in near stasis as summer has turned to fall. 

The two leaders – Biden and Warren – are the two candidates who have presented the clearest rationales for their candidacy.  Biden fundamentally promises a return to the Obama years and Warren pledges big structural change.  The latter is making some observers nervous, resulting in a spate of polls that show general election voters are not yet ready to embrace big structural change.

The most recent NBC/Wall Street Journal poll shows a plurality of non-Democratic primary voters supporting smaller scale policy changes and majority opposition to some of Warren’s policy proposals.  The centrist Democratic organization Third Way presents data that voters want a more centrist approach on health care rather than Medicare for all.  CNN continues to show that voters prefer a candidate who can defeat Trump over one with whom they agree on the issues, which may be a false choice if they can have both.   

There are many reasons to be anxious about the 2020 election.  The stakes are extraordinarily high, we are now in an impeachment process, and, with over a year to go, many factors are simply unknowable, including the progress of Democratic candidates as they move toward the nomination and the general election, the erratic behavior of the president, and the potential for corruption of the process.

I am not, however, concerned about Warren’s articulation of the need for big structural change.  Here’s why:

  •  Warren has left herself a lot of room to define the nature of structural change.  The words establish her as the change candidate, and as a clear contrast to Biden’s return to the recent past.  As the leading woman candidate, and a Biden alternative, she would represent change in any case. Embracing that positioning seems smart and many of her proposed policies, like increasing taxes on the super wealthy, are in fact broadly popular.
  • Warren has the capacity to be a reform candidate. She is financing her campaign differently than the other candidates, and she is undaunted by demands of both big corporate interests and the super-wealthy. For the 30 years I was in polling, messaging of standing up to big corporate interests to bring change has been a strong elixir. Back in 1990, in polling for the late Senator Wellstone (who, for the record, was always clear he didn’t listen to his polling), 72 percent of Minnesotans said the problem in Congress was more that its members listened to special interests than that problems were beyond government solution. Similar results have replicated in the interim but few candidates can authentically articulate the message. Despite two Pinocchios from the Washington Post, Warren is uniquely able to articulate that her presidency would listen and respond to people and not to special interests (hopefully combined with a plan for economic growth and small business development). Genuine reform in how we conduct business in Washington would be big structural change.
  • Voters will likely be more interested in the results than in the process of change.  Warren has a variety of plans – and ways of paying for them that do not require tax increases on the middle class.  Voters favor lower health care costs, more accessible post-secondary education, more economic opportunity, fair treatment and fair pay in the work place, and Warren is talking about these issues.  Voters are not – at this point – ready to embrace Medicare for all but they may also understand that it won’t happen unless they do and there are interim steps in the process they may endorse moving forward. 
  • The impeachment process may change the context.  On the downside, it may make Washington and Congress look even more partisan and angry.  On the upside, it may focus discussion of the threats of the Trump presidency.  Democrats have so many complaints about Trump that our attacks are like spam – diverse, diffuse, and occasionally obscure to some people.  That he represents a threat to national security and to the electoral process in which people choose their own leaders can become central to arguments against him.  In either case, the process may spur greater interest in change from business as usual in Washington even if the desire for change encompasses both parties.     

None of this discussion should suggest I do not have anxieties about the leading candidates.  My principal anxiety about Warren is whether she will appear the Harvard professor who needs to be the smartest in the room, or whether she is the woman of blue collar roots motivated by instincts of caregiving and reform.  Candidate imagery and gender interact, and I am sure her campaign is well aware of the image downsides of being the “Smart Girl.”  As for Biden, his strength is in a perception that he is a known quantity and a decent man, who represents little that is radical or risky.  Other than gaffes that can undermine perceived steadiness, I worry that he will not connect with younger voters whose heightened participation is essential to prevent this electorate from being older than the 2016 electorate, a demographic change that would favor Trump.        

Additionally, Sanders may garner more support than I am crediting him with here and others may emerge.  There is room for both to happen.  A three or four candidate late field can spur another anxiety:  that no one have a majority of delegates going into the convention. 

I am not, however, anxious that Warren is the candidate of big change.  If the country moves from Trump to Warren, it will be a big change – in structure, process, and result. 

Shortly after the 2016 election, I had lunch with a colleague whom I respect.  I noted that in politics as well as physics, for every action there is a reaction.  We went from a brilliant, erudite President who believed in meritocracy to the current incumbent.  The next wave, I suggested, could bring big change.  Maybe, my colleague responded, but we are in for a whole lot of hurt in the meantime.  His prediction was correct.  We will see if mine was as well.    

Is Joe Biden the most electable Democrat? Maybe – but maybe not…

Joe Biden’s lead in the Democratic primary field apparently rests on the three legged stool of long-standing familiarity, appreciation particularly among older African American voters as President Obama’s trusted second, and a perception that he is more electable than other Democrats. 

Electability is not actually a testable proposition as the Democrats will have only one nominee who will win or lose.  Still, the concept of electability is important to voters and thus worth examining.

Biden’s perceived electability may rest in part on his having been part of a winning national ticket.  But as Natalie Jackson pointed out in the Huffington Post, of the nine Vice Presidents who ran without first acceding to the office on the death of the President, only three have won.  George H.W. Bush was the most recent Vice President to win, then Richard Nixon, although he lost before he won.  Before Nixon, you have to go back to Martin Van Buren to find an example of a Vice President who won election without becoming an incumbent first (Vice Presidents Elected President). 

Blue collar appeal – or the avoidance of elitism and its imagery – will certainly be important in the 2020 election, given the states needed for an Electoral College win. Biden has a reputation for appealing to blue collar voters, although the evidence seems largely anecdotal.  He grew up in a blue collar family but such roots are not unique in the field. They are shared by at least Elizabeth Warren and Bernie Sanders.  Others grew up in families that were far from wealthy. Several are the children of school teachers or of a single parent who struggled financially.  It is not a field of elites.   

Here are some other considerations where Biden may not reach parity with others in the field:

A Referendum or a Choice:  One of the most fundamental framings of an election is whether it is a referendum on the incumbent or a more lateral choice between the contenders.  The Democratic nominee will be the challenger to an unpopular incumbent.  Democrats should be advantaged if the election is a referendum on Trump. 

Biden may be the best positioned to make the campaign a referendum because he is more moderate, has lower negatives (at least now), and is a comparatively comfortable choice for many. 

There is, however, a risk that the electorate is not in the mood for such a referendum election.  They may want to know what is next on raising wages, lowering health care costs, making America safer, healing divisions, and changing the way we do politics.

Biden may have answers on all of those issues, but his long record can be a hindrance in saying that he will bring more positive change than the country experienced in the Obama administration.  As popular as President Obama is, the demand for change may exceed the nostalgia for him.

A Different Electorate.  If the electorate were similar to the 2016 electorate, a return to the Obama years might be enough.  The 2018 election saw enough dissatisfaction with Trump, and enough change in turnout patterns with higher Democratic and lower Republican participation, that it is tempting to simply try to carry the 2012 – or 2018 – election forward.   

The electorate seems unlikely to be a mere extension of the last two elections, however.  The eligible electorate will have more voters of color, especially voters of Hispanic origin, and it will be older, absent a shift in turnout that increases the number of younger people who participate. (Pew: An Early Look at the 2020 Electorate).

The 2020 electorate promises to be larger than the 2016 electorate.  The Trump campaign and its allies are already spending millions online to find people who have not voted before or voted irregularly whose participation they can compel.  Emotion on the Democratic side likely means higher turnout organically – and presumably Democratic efforts to expand the electorate will eventually match Republican efforts.

Still, an electorate that is more Democratic than 2016 likely depends on turnout, particularly among younger voters who have the most room for turnout growth.  It may be that an anti-Trump message will compel turnout, but it would also be useful to have messaging and a candidate who can optimize that young voter participation. 

Biden does not appear the best candidate to motivate younger voters.  Younger candidates, particularly Pete Buttigieg and Julian Castro are speaking to generational change, while Jay Inslee is making climate change – an issue particularly compelling to young voters – central to his campaign, and Elizabeth Warren is speaking to student debt and child care issues, which resonate with younger voters. 

Indeed, on the message level, several other candidates seem to have stronger youth appeal than Vice President Biden.   

Anti-Partisan Voters.  Swing voters by definition are not locked into a party – that is part of what makes them swing voters – and the voters who supported both Obama and Trump fall into this category.  Biden seems to be betting that they will like bi-partisanship, and speaks to his historic civility with Republicans. 

Many swing (and third party) voters, do not like either political party and are more interested in non-partisanship than bi-partisanship.  They are looking less for politicians to “reach across the aisle” (a term they often do not understand) than for leaders to be separate from either party.  That is a tougher case for Biden who has lived party politics, than for those like Governors and Mayors who have some separation from the hyper-partisanship of Washington, or those who are newer to politics than Biden is. 

Swing voters tend to be younger than average and are disproportionately women.  Biden may not be ideally positioned to motivate either young voters or women voters who are mistrustful of politicians of either party.  There are white, blue collar, older men who voted for President Obama and for Trump, but not very many and they are unlikely to be the core swing vote, as opposed to younger women without college experience – across ethnic lines – who are low or moderate propensity voters.     

# # #

The first test of electability is whether a candidate will be nominated.  Biden has the lead right now in most polls, although the size of the lead depends on assumptions about the shape of the electorate.  Biden is stronger among older Democrats than younger Democrats, and seems especially strong among older African American voters. 

Biden seems for many Democrats to be the safe choice at a time when defeating Trump is Priority One. The problem is that playing it safe may not be the best winning strategy.

The first test for some of the contenders with apparent advantages among younger voters, is whether they can motivate their participation in the early primaries and caucuses.  If so, they have more room to overtake Biden.  In showing their capacity to attract younger voters, they may also take away his electability argument.