A list of the things that could break election models
Voter suppression, a check-shaped recovery and a giant meteor are all things I’m worried about (to varying degrees...)
A half-rant, half-blog post for you today:
I’ve notice this tendency for Very Online People to object to election models because they “don’t factor in voter suppression and Russian interference.” I get the spirit of the objection and can make some (slightly informed) guesses about where it’s coming from in the Very Online Twitter Brain, I think.
My theory goes like this: 2016 was a big surprise for lots of people. For many online and on the left, the idea that enough Americans would vote for Donald Trump to put him in the White House was a huge break from their lived experiences and expectations for their fellow citizen. Such a worldview-shattering event has caused them to look out for information that may have led them astray. Polls are an easy boogeyman (boogey…Data??) so get a lot of blame that could be spent on revisiting our own thinking.
Well, to be fair, polls were wrong in a few key states. And they were more wrong there than average. But they weren’t earth-shatteringly wrong. The 4 point error in Wisconsin wasn’t even a 2-sigma event (IE it was within the margin of error for polls). So, should we really be heaping on a bunch of conspiratorial thinking for why they were wrong? Can’t we just accept that the methods were flawed and… move on? Acknowledging the flaws is the only way to fix them, after all.
And to be fair on another account, the concern over vote suppression—both intentional and unintentional (eg from coronavirus, bad planning, shark attacks)—is a pretty real one. Studies of turnout in Georgia, for example, have found that depressed turnout directly impacted Democratic candidates’ fortunes, even though it didn’t change the outcome of the election.
But is it fair then to jump from saying that (a) voter suppression is a (bad) thing that exists to (b) it will cause polls to misfire so badly that you shouldn’t trust the models? That’s where I’m drawing the empirical line. At best, we have evidence that voter suppression has a marginal effect in a few states. That is… not enough to discredit the polls, and certainly not enough to discard all our forecasts.
For posterity’s sake I am left wondering, what could break our election models? It might be helpful to just list a bunch the factors that aren’t expressly modeled by most of our forecasts, either because we can’t quantify them or they don’t fit in our statistical distributions
Coronavirus causes turnout to drop 10% across the board and pollsters don’t model it properly
Widespread adoption of mail-in ballots causes turnout to increase by 10% across the board and pollsters don’t model it properly
There is a severe disconnect between the state of the economy and the election—beyond what we can model by interacting the state of the economy with a measure of political polarization
An unforeseen event (meteor, a second pandemic, a world war? What else?) causes polls to move in the last few months by a magnitude we have never, ever seen before
And again for posterity’s sake, here is a list of things I don’t think our models can or should be taking into account:
The chance that Trump “refuses to leave office” (whatever that means)
Shark attacks
Foreign interference exposes one side of the aisle to a bunch of fake news (polls would catch this)
Someone hacks voting machines and changes actual votes (not a thing that happens)
Here’s a related post from Andrew Gelman on the uncertainty in the election model we built for The Economist.
It’s reasonable to believe the polls, especially if the model you’re looking at takes into account the added uncertainty from the economic collapse. But honestly, elections are pretty predictable, and small changes in voting habits don’t usually break our models beyond the uncertainty we can quantify already. I, for one, believe the data.