This is the first of a multi-part series on polls and how the news media serially misrepresents them, as well as ways to think about them that I - famously not a pollster, but someone who works in voter sentiment research - have found helpful.
It was noted political analyst and regular MSNBC contributor Herman Melville who admonished us, 173 years ago, to look not too long into the face of polls, and he was right. His argument was that when we stare too long into polling, we become transfixed by strange images, beguiled by the shifting shadow shapes of fiends that seem awful and significant to our rapt gaze, but which are, in fact, ephemeral. The fiends and shapes and visions are not in themselves a danger to us; it is our fixation on them, which draws our attention away from precarious and essential tasks at hand, that can lead us to ruin.
Literary critics will tell you that the relevant passage from Melville was written 85 years before the advent of scientific polling (not to mention 145 years before MSNBC) and was, in fact, about gazing too long into the fire while steering a ship at night. LIES! Falsehoods told for the purposes of deception and base personal gain! You hate to see it.
But let us leave the lamentable moral condition of the modern literary critic to one side. Melville’s warning is a fair one; staring too long and too hard into the polls that are released on a near-daily basis during this time of the election is an invitation to demonic visions of the worst sort. But this admonition calls a natural question: what, then, are we to do with polls?
On this, Melville is conspicuously, indeed suspiciously, silent. It’s an area on which he’s gotten a free ride up to this point and I’m damned if I’ll stand for it a moment longer.
And so, as has often been the case throughout my career, I am once again called - by duty and public acclaim1 - to rectify the omissions of the noted political analyst Herman Melville.
**
The contemporary fixation on polling is entirely understandable. The human brain does not particularly care for ambiguity. To violently compress reams of scientific literature (as is my wont): the brain likes for the world to make sense. It wants important variables - especially matters that will affect material security, or that concern identity; politics does both - defined. (Consider the staggering amount of time, energy, and devotion that humans have poured into defining the fundamentally-indefinable variable of what happens after death.) Once defined, those variables can be put through the predictive systems our brains build to give events context and cause: “this happened or will happen, and that is how to explain it”.
Ingesting polls feels like an opportunity to define a variable: who will win the election? This tendency, I think, was exacerbated in 2008, when two things happened: a majority of the country wanted a change after the Bush years, and a guy blogging under the handle Poblano (later revealing himself as Nate Silver) started to make predictions, based on polling and demographic analysis, so accurate that they suggested some kind of second sight. The people badly wanted something, and a wizard turned up out of nowhere and told them they would get it, and lo it was so.
There are other factors, of course, but I think most stem from those two, and 2008 ushered us into a new era of Celebrity Nerds Who Can Tell The Future.
Until, of course, two more things happened in 2016: polling, as an industry, got the election wrong (although not, I would argue, as catastrophically as is generally perceived), and our national politics entered a phase more explicitly cruel and dangerous than many Americans could have thought possible. 2016 thus raised both the uncertainty around elections and, at least for many Americans, the stakes (emotional and practical) of their outcomes.
The result, of course, was that a sizable subset of people absolutely lost their shit. In the years following 2016, I was variously told, by individuals with political qualifications ranging from regular MSNBC viewership to positions of authority and influence within the Democratic Party, that pollsters should not be paid if their polls are incorrect; that the entire industry of polling could never be trusted again; that analytics (that is, predictive modeling of voter behavior based on a variety of inputs, which is not polling but sits next to it) should be discarded wholesale; that media should be banned for discussing polls; that private polling firms should make all of their polls and methodology public (which, for what it’s worth, would ruin them overnight); and that polling should be nationalized (I did not attempt to lead that person in a rousing rendition of The Red Flag and bitterly regret it).
The problem, of course, is not with polling or pollsters. Oh, they got 2016 wrong, almost all of them, to varying degrees. Many got 2020 wrong, too, but less so. Some of that was about correctable mistakes, but the reality is that the electorate was - and is - unsettled, roiled by a number of factors related to the breakup of the Reagan Political Compact that has dominated American politics and government for the last four decades. The best math in the world won’t save you if people won’t engage with polls or decide to vote in strange ways or both.
The fault is in how polls are presented and covered. The modal way that a poll appears in media at present is a contextless headline (SFA POLL: SPRING UP 3 ON MELVILLE) followed by a presentation of a finding or two (“Spring has a lead amongst intolerable pedants, while Melville is performing well with duplicitous literary critics”), followed by an analyst saying the election is a toss-up. This is, technically, an accurate way to present a poll.
It also does a disservice to both the audience and the poll, because it misses what I would argue are the two most important questions of any poll: who is moving? That is: what segment of the electorate is changing its mind? And, equally importantly, why is that segment moving? Without those two pieces of information, polling coverage is just “A number went up! The election is still close!” which effectively tells the reader nothing.
And, again, the human brain wants the world to make sense; it craves defined variables, and context for those variables. “Number up! Election close!” just doesn’t cut it because it doesn’t offer any explanation for why that is happening. And here we come to the heart of it:
Polling can certainly tell you who is moving within the electorate. But polls are not designed to tell you why, and they should not be asked to.
To be clear: an individual pollster can certainly hazard an informed speculation about why a segment of the electorate is changing its views (or not), and very well might be right. But polls, as a tool, are not well-designed to reveal motivations. They can make a stab at it; the standard model is to ask what issue voters most care about, and which candidate they trust with that issue more, and draw the relevant conclusion. There are more sophisticated versions of this using more variables, of course, but that’s the essential practice.
The problem is that that is not the only way people decide how they will vote; in fact, it isn’t even the predominant decision-making model. If voters cast ballots for candidates based on their feelings about specific policy issues, Democrats would control both chambers of the federal legislature and a majority of governorships and state legislatures. As we’ve seen with the results of plebiscites all over the country, Americans - from a strictly policy perspective - are a lot more progressive than is generally understood.
Rather, voters are more likely to make their decision based on which candidate they identify with - or against. This feeling (and it is a feeling) derives from a mass of factors, mostly rooted in the perception of a shared lived experience or the sense that a candidate at least understands and values you and your lived experience.
Issues and identity can overlap; they often do. And sometimes one can appear to be the other. Consider abortion. Abortion is polled as an “issue” - how important is it to you; do you believe it should be legal in any circumstances, some, or none; which candidate do you trust more/most on it; etc. The answers to those questions are all useful! But to be a voter who casts their ballot on the “issue” of abortion, in opposition to abortion bans and the candidates of the party that would impose them (the GOP) is less about saying “I might want/need an abortion in the future” and more about saying “women are in fact people and should be treated as such, and additionally: fuck you and the horse you rode in on”. Issue, or identity?
So when a segment of the electorate moves toward one candidate or another, or away, what we’re seeing is some portion of that segment is identifying more (or less) with one candidate or another. Unless we know, or can reasonably speculate, about why, we have no idea if this movement is real, or how durable it is likely to be, much less what to do about it.
“Number Goes Up! Election Close!” presents the American electorate as one big mass of people, an unclear portion of whom periodically change their minds for unclear reasons. It’s not a failure of polling, it’s a failure of coverage. And it doesn’t help us answer the question: who, really, is still changing their minds? What voters are still technically up for grabs?
More on that in my next one.
no