So, I just got off the phone with an old friend who is on the communications side of political consulting. My friend is apparently giving my former polling colleagues a rough time and apparently so are others – “suggestions” that are not feasible, organizations that are assigning them letter grades as they would to school children, clients dismissing the need for research at all.
Now, I have been pretty clear in these pages that I think polling as a methodology no longer works the way people think. It is a rougher measure and can leave important groups out of the equation. It has value but it also has serious limits: It is far less useful than it used to be for prognosticating close elections. Low response rates allows greater risk of response bias and, as a result, sampling is more complex.
Polling risks leaving out constituencies that may be critical to winning – voters who are anti-establishment (or anti elite) and see it as an elite or establishment tool, and those who just don’t relate to the political frame as employed. Except perhaps for this last one, none of this is the fault of pollsters, and imposing the extant political frame on swing and low propensity voters who aren’t interested in it is hardly an error unique to pollsters. The Washington political frames to which many voters do not relate is a shared Washington responsibility.
Here’s what I think are actually the remedies to better political research by campaigns:
1. More upfront strategic thinking about how to win. There is a plethora of information available for any district or state, including prior election results, demographics and analytics, and two (or more) real candidates with unique strengths and weaknesses. After studying all that, what are the hypothetical ways to win that you need to test? (Chances are there are better methods than polling for choosing which is most likely.)
2. Better analytics and better integration of them strategically. Political analytics got better and better from 2006 through 2012. Then its practitioners started competing on cost and cutting corners on what they did statistically. At the same time, people seemed to think it was a good idea to separate analytics from the process of campaigning so it was an independent look and not integrated into the campaign process. Both of these developments were unfortunate in my view. Cutting corners made analytics less valuable as a predictor and the separation from campaigns meant than campaigns did not have the capacity to ask for a sophisticated statistical look at the challenges that were on the table strategically. It’s time to go back to the future on analytics – an invaluable tool that should be guided strategically.
3. Tailored research that answers the strategic questions on the table. In close elections, winning is often on the margins. Hypothetically, maybe your candidate can win if you can move 6 percent more of Latino voters, or lose a particular suburban community by a little less, or find a way to blame the incumbent for the serious infrastructure problems in a community that usually votes for that person’s party. Strategic analysis and analytics can help you develop these options. There are experiments you can conduct to say which one(s) might help put your candidate over the top. And a poll of voters in the aggregate wont tell you which one will work anyway.
4. Integration of field data into research. Almost any good campaign has a field program in which people go talk to individual voters, including those who are swing voters and lower propensity voters. I don’t want to mess up the open ended nature of these conversations, which is part of what makes them valuable, but there are ways of capturing quantitative information from them – and that is about the only way you really will hear from genuinely non-partisan and non-political voters, and those who vote irregularly. You have to start the field program earlier, but that is generally a valuable thing to do for other reasons.
So, yes, there are new challenges in political research. The biggest problem in polling is that you can no longer talk to people at random because they don’t respond at random. Careful polling makes that less of a problem and sloppy polling makes it worse but it is not feasible to eliminate the problem. The problem is the result of caller ID, telemarketing, political polarization, and changing modes of communication. The pollsters did not create the problem.
Generally, pollsters are analytic and political thinkers with a penchant for numbers. Those skills sets are important in the mix of campaign skills. Conversation about methodology is useful. Creativity on how to answer strategic questions is essential. Increasingly, the presence of advanced statistical skills on the team is important. Beating up on the pollsters won’t help to find new and better ways to conduct research.