This is the worst way to cover the polls. Please stop
Binary reports ("x is up, y is down") are perhaps the worst way to communicate polling information
Here’s a message I almost tweeted this morning until I reminded myself that I’m trying to stay off Twitter for a while:
Every four years is a chance for people to learn a lot of useful things about how to report on polls, and sadly you can count on them letting you down every time.
After I wrote this newsletter I also drafted another tweet, but decided to commit to the ban and not to post it:
I have seen a lot of reporting on the recent georgia polls that don't any information about margins of error or uncertainty. Have y'all learned absolutely nothing?
All this rage was prompted by the following article published in the National Review yesterday and which I just so happened to see this morning:
After an election in which polls saw a clearly above-average prediction error, I cannot believe people are still covering polls in the binary fashion — “Democrats are up, Republicans are down.” This is especially infuriating when the poll shows leads for both Rafael Warnock and Jon Ossoff well inside the margin of error (sampling + non-sampling) for a poll of 583 likely voters!
The piece doesn’t even mention the margin of error. For shame.
…
Okay, here’s a related and more sophisticated point about that poll. Vox covered it with the headline “A new Georgia poll suggests that Republican lies about “voter fraud” are hurting the GOP” and subhead “Trumpy calls to boycott the election could help throw Senate runoffs to the Democrats.”
The evidence for this is very week. They make their case by linking to a tweet from Nate Silver in which he says the poll implies there has been drop-off among Republican voters because the sample for 2020 past vote is Biden +4 and he only won by 0.2.
That might not be evidence for GOP drop-off, though. Equally plausible is that the poll is having a hard time reaching Trump voters so its sample is skewed toward Democrats. (You know, that thing that just happened last month?)
If I were y’all I would just be throwing my hands up re: Georgia at this point. The race is too close to know definitely who is ahead. Sure, polls can give us some idea, but as one analyst said to me recently, at this point we’re really using a magnifying glass for a problem that needs a microscope. Call it a close race and move on. That’s really all the polls can definitely tell us.
EDIT: Another explanation is that the National Review journalist isn’t actually stupid but wrote the piece bad faith to increase GOP turnout in Georgia. Which, yeah, I guess that’s likely.