Nearly every time Bernie Sanders wins a primary or caucus, it comes as a surprise to pollsters. In Minnesota, Oklahoma, and Kansas, all states Sanders won, polls had Clinton out ahead by at least a small margin, and in some cases a very large one. Even his virtual tie with Clinton in Iowa was unexpected. But of all the surprises Sanders and his supporters have brought to this cycle, last night's big Michigan win was by far the biggest.
Just look at this chart:
X content
This content can also be viewed on the site it originates from.
Throughout this election cycle, Sanders has been perpetually underestimated. That's not an opinion. It's a fact. Just as the media easily dismissed Donald Trump's campaign as a sideshow to the Republican party's main event last summer, it did the same to Sanders. The only difference was, the press gave Sanders a fraction (or a fraction of a fraction) of the same coverage. The polls predicted that ignoring Bernie was okay, though, because it looked unlikely Sanders would win any primary states aside from New Hampshire and Vermont anyway.
But win he has. And not only has he won: starting early this year, he began out-raising Clinton month after month. Still, the polls kept missing this momentum. Last night in Michigan, they missed big time.
The fact is, these misses are no fluke. Instead, it seems that pollsters across the board have something of a Bernie blind spot. One key reason for that—though not the only reason— is that Sanders supporters are overwhelmingly young, and lots of pollsters still haven't found a way to reach young voters where they are: their smartphones.
Polls have traditionally been conducted on landlines, not only because that was the prevailing method of reaching people for decades, but also because recent regulations have made it illegal to robocall cell phones, a practice that's still allowed for landline polls. That makes polling cell phones far more expensive to conduct, because it requires more human labor.
While larger operations like Pew Research and Gallup have moved predominantly to cell phone polls, other cash-strapped operations, forced to churn out an ungodly number of polls to accommodate the news cycle, often defer to landlines. That may explain, for instance, this landline poll from a Detroit Fox affiliate that shows Clinton trouncing Sanders in Michigan 66 percent to 29 percent. The pollster for that report justified the methodology, writing, "Because likely Primary voters are older, 54 percent are 60 or older and 86 percent are older than 50, we believe there are sufficient land line voters to get an accurate sample."
But that logic is flawed. By excluding cell phones, landline polls risk alienating a huge percentage of voting-age adults, says Courtney Kennedy, Pew's director of survey research. "With Sanders, in particular, if you don't have a sample design that's going to get you a representative share of that younger population, you'll be in trouble," Kennedy says.
What's more, these polls tend to have a domino effect, because aggregators like RealClearPolitics and even the esteemed FiveThirtyEight use polls as a basis for their predictions. Flawed poll designs, Kennedy says, can throw those predictions off wildly.
The chief problem, as she sees it, is that some of the standards that long governed the polling industry have faded away as pollsters try to adjust to the changing technological landscape. And the breathless media coverage of the campaigns, Kennedy says, doesn't help.
"In today’s media environment, pretty much everything gets picked up," she says. "There’s basically no standards the way there used to be to vet and weed out polls. So you see some of these polls where you just take one look at the methodology statement, and it’s obvious there are fundamental flaws."
Of course, landline polls can't be blamed entirely for the upset in Michigan. After all, as Frank Newport, editor-in-chief at Gallup points out, there were plenty of legitimate polls conducted by cell phone that got Michigan wrong, too. Primary races, Newport says, are just notoriously tough to predict. That's especially true in Michigan, where people are free to vote for any party, regardless of their registration. That, Newport says, can draw independent voters to the ballot box, as well.
Mark Blumenthal, head of election polling for SurveyMonkey agrees. "Unlike general elections where most people are rooted in their parties, in primaries you're picking from a lot of candidates you like or could conceivably vote for," he says. "There's a lot of late movement."
Another hiccup for pollsters is modeling the demographics of the likely electorate on primary day. "It's a much smaller turnout universe," Blumenthal says, "So getting the electorate right is a bigger challenge."
Considering how stark the demographic differences between Clinton's base and Sanders' base are, "getting the electorate right" is even more critical this cycle. Broadly speaking, when turnout among black voters is high, Clinton has the advantage. When turnout among young voters is high, Sanders does. Last night, it was the latter, with 18 to 29 year olds making up 21 percent of the electorate. A whopping 81 percent of those voters went for Sanders. Any polls that underestimated the youth voter turnout, Blumenthal says, would likely have seen skewed results.
Whatever the reason, though, the fact is, these problems aren't going away, and it's unlikely pollsters are going to change their methodology drastically in the middle of the primary season. Which is why, Kennedy says, we shouldn't be surprised if similar upsets arise in future primary races. "If you see some of these same designs used," she says, "it's very possible you could see a similar situation where support for a candidate was missed in some of the polls."
But don't take her word for it. On a call today about last night's primaries, Clinton campaign manager Robby Mook told reporters, "We would all be well advised to treat public polling coming out today and this week with skepticism."
