Looking at the wrong problem

So, I just got off the phone with an old friend who is on the communications side of political consulting. My friend is apparently giving my former polling colleagues a rough time and apparently so are others – “suggestions” that are not feasible, organizations that are assigning them letter grades as they would to school children, clients dismissing the need for research at all.

Now, I have been pretty clear in these pages that I think polling as a methodology no longer works the way people think. It is a rougher measure and can leave important groups out of the equation. It has value but it also has serious limits: It is far less useful than it used to be for prognosticating close elections. Low response rates allows greater risk of response bias and, as a result, sampling is more complex.

Polling risks leaving out constituencies that may be critical to winning – voters who are anti-establishment (or anti elite) and see it as an elite or establishment tool, and those who just don’t relate to the political frame as employed. Except perhaps for this last one, none of this is the fault of pollsters, and imposing the extant political frame on swing and low propensity voters who aren’t interested in it is hardly an error unique to pollsters. The Washington political frames to which many voters do not relate is a shared Washington responsibility.

Here’s what I think are actually the remedies to better political research by campaigns:

1. More upfront strategic thinking about how to win. There is a plethora of information available for any district or state, including prior election results, demographics and analytics, and two (or more) real candidates with unique strengths and weaknesses. After studying all that, what are the hypothetical ways to win that you need to test? (Chances are there are better methods than polling for choosing which is most likely.)

2. Better analytics and better integration of them strategically. Political analytics got better and better from 2006 through 2012. Then its practitioners started competing on cost and cutting corners on what they did statistically. At the same time, people seemed to think it was a good idea to separate analytics from the process of campaigning so it was an independent look and not integrated into the campaign process. Both of these developments were unfortunate in my view. Cutting corners made analytics less valuable as a predictor and the separation from campaigns meant than campaigns did not have the capacity to ask for a sophisticated statistical look at the challenges that were on the table strategically. It’s time to go back to the future on analytics – an invaluable tool that should be guided strategically.

3. Tailored research that answers the strategic questions on the table. In close elections, winning is often on the margins. Hypothetically, maybe your candidate can win if you can move 6 percent more of Latino voters, or lose a particular suburban community by a little less, or find a way to blame the incumbent for the serious infrastructure problems in a community that usually votes for that person’s party. Strategic analysis and analytics can help you develop these options. There are experiments you can conduct to say which one(s) might help put your candidate over the top. And a poll of voters in the aggregate wont tell you which one will work anyway.

4. Integration of field data into research. Almost any good campaign has a field program in which people go talk to individual voters, including those who are swing voters and lower propensity voters. I don’t want to mess up the open ended nature of these conversations, which is part of what makes them valuable, but there are ways of capturing quantitative information from them – and that is about the only way you really will hear from genuinely non-partisan and non-political voters, and those who vote irregularly. You have to start the field program earlier, but that is generally a valuable thing to do for other reasons.

So, yes, there are new challenges in political research. The biggest problem in polling is that you can no longer talk to people at random because they don’t respond at random. Careful polling makes that less of a problem and sloppy polling makes it worse but it is not feasible to eliminate the problem. The problem is the result of caller ID, telemarketing, political polarization, and changing modes of communication. The pollsters did not create the problem.

Generally, pollsters are analytic and political thinkers with a penchant for numbers. Those skills sets are important in the mix of campaign skills. Conversation about methodology is useful. Creativity on how to answer strategic questions is essential. Increasingly, the presence of advanced statistical skills on the team is important. Beating up on the pollsters won’t help to find new and better ways to conduct research.

Author: dianefeldman1764

In December of 2018, I closed down the polling firm I operated for nearly 30 years. I continue to consult and write on research and politics, while living here in Jackson, Mississippi. Jackson is on the Pearl River and so I named the blog View from the Pearl. All views are my own, newly unfettered from the need to run a polling business or please anyone. Please click Follow to receive posts in email.

4 thoughts on “Looking at the wrong problem”

  1. A good article which could easily be titled, Realistic Expectations for Polling in the 2020’s. I especially liked suggestion 4. Maybe you could do a blog on your vision of how that can be done in a meaningful way.

    Like

Leave a Reply to Frances Smith Strickland Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: