Are polls overestimating the Democrats again? 📊 June 14, 2020
Early signs suggest data in some states could be too pro-Democratic, but most polls look fine
Welcome! I’m G. Elliott Morris, a data journalist and political analyst who mostly covers polls, elections, and political science. Happy Sunday! This is my weekly email where I write about news and politics using data and share links to what I’ve been reading and writing.
Thoughts? Drop me a line (or just respond to this email). If you like what you’re reading, tap the ❤️ below the title; it helps the post rank higher in Substack’s curation algorithm. If you want more content, I publish subscriber-only posts 1-2x a week.
Dear reader,
I apologize to those of you who look forward to my weekly posts that I have not sent one out for two Sundays in a row.
However, I do not return from the hiatus empty-handed. The Economist has now officially launched the election forecasting model I’ve been working on over the past few months. It’s a pretty ambitious forecast; a fully Bayesian polling aggregation model written in Stan (thanks to Andrew Gelman and Merlin Heidemanns of Columbia University) that also takes economic information into account. We’re proud of it and hope it will teach you something!
I’ll flag one complication now in how I talk about the forecast: Per usual, I am trying to keep my work at the paper and this newsletter as separate as possible, so I’ll never be writing explicitly about the forecast here and will be referring to public information about the model in the rough third person. This will cause some odd sentence structure sometimes but I hope you can appreciate the need for the convention.
With the launch of The Economist’s forecasting model, the 2020 election has begun in earnest. What an exciting time for data nerds like us.
Are polls overestimating the Democrats again?
Early signs suggest data in some states could be too pro-Democratic, but most polls look fine
Odds are that you’re scarred by the polls “missing” Donald Trump’s victory in 2016. That’s not an irrational position to hold. After all, while polls are still useful, they are not perfect. If anything the last election was just a reminder of that fact. Polls were pretty good in 2008 and 2012—better than average, in fact. That may have fooled us into a false sense of complacency in trusting the data.
I will set aside for now the fact that good models (such as The Economist’s) gave Trump a pretty good shot at winning the election. When read literally, the margins of error surrounding the forecast said that a very normal polling error would have been enough to elect him. That is exactly what happened! The polls may have missed the outcome, but good forecasts could have given you a good idea for just how likely that was to occur.
This has got me thinking over the past couple of weeks whether are polls overestimating the Democrats again. Are we doomed to repeat 2016? Now that election forecasts are coming out again, this is a question of high public import.
My early analysis suggests not. Many (not all) pollsters have fixed the issues that caused their data to misfire in 2016 (weighting by education, mostly, but also choosing good LV models). But more than that, the polls today look pretty sensible when compared to higher-quality benchmarks.
Take The Economist’s work as an example. I download their estimates today and compared them to the MRP estimates they published a few months ago. They match up pretty darn well:
The match is not perfect, however. Polls in Michigan and Ohio are especially more Democratic than the MRP benchmark (which is essentially equivalent to a poll weighted by education, race, party etc—the whole shebang) and still a little “too” blue in North Carolina and Florida. But in other states, like Texas, Arizona, Wisconsin and especially Georgia, they look too red. This to me indicates that the industry is not systematically biased toward the Democrats.
Though that is no guarantee that they won’t be in the future.
Posts for subscribers
June 4: Trump is doomed! Biden is doomed! Trump is back on top! Wait... Pump the breaks everyone. You have to take a holistic, long view of the data.
June 8: How to think about “unskewing” the polls. Pay no attention to the pollster behind the curtain.
What I'm Reading and Working On
I’m on vacation right now so I’m not reading or working on anything. Thank god. But in the off chance that you do have something to recommend to me, I’m all ears (or all inbox… as in, email me).
Thanks for reading!
Thanks for reading. I’ll be back in your inbox next Sunday. In the meantime, follow me online or reach out via email if you’d like to engage. I’d love to hear from you!
If you want more content, I publish subscriber-only posts on Substack 1-3 times each week. Sign up today for $5/month (or $50/year) by clicking on the following button. Even if you don't want the extra posts, the funds go toward supporting the time spent writing this free, weekly letter. Your support makes this all possible!