I don't think we have anyone in the mainstream media, talking about polling the way it should covered. This means that the inaccurate conventional wisdom takes on polling remain.
In 2012, the conventional wisdom was that polls were imperfect, we shouldn't trust them. The polls were pretty accurate and Obama got reelected fairly easily. The general reaction was that we should trust polls.
In 2016, the conventional wisdom was that polls were perfect, but a normal polling error occurred and Trump got elected. The general reaction was that we shouldn't trust polls.
In 2020, the reaction to the polls being off again, have led to reaction that we should never trust polls again.
Maybe it was just people's bias talking, but I saw a lot of takes that Biden would win by double digits and win 413 Electoral votes. I personally thought that was a bit on the crazy side. Biden winning by double digits and Texas seemed unlikely given polarization. I think this may have caused the reaction of "we should never trust the polls again". The "blue shift" effect may also have contributed to that narrative as well since Trump was leading in MI, WI, and PA by a lot on Election Night.
I think the last point here is a very good one. Many of the pieces about polls and forecast were written on election night when the incomplete returns looked much different (redder) than the full returns, but takes have not been rewritten. I would like to see everyone who wrote about the polls from Nov 3rd through 7th reevaluate their takes.
Hello Elliott, If the margin of error generally reported only reflects sampling error and the “true-er” margin of error is really double or more than the sampling error that means typical state polling error is 6 to 10 to maybe 13 percent. It would seem then that polls would then be “reliable” only in states that knowledgeable politicians would be able to call without polling. Meaning it is likely that Utah goes Republican or DC goes Democratic. In states that are in the Lean Dem, Lean Rep or Tossup states to use Cook Political report categories polls provide little directional insight. Do I have that right?
It depends on what you mean by “reliable.” Remember that errors at the tail end of the margin of error are still rarer than errors nearer to the center (0), so if a candidate is up 5 in a poll it’s still likelier that they win than that they lose. But if we tell people that the margin of error on any of those polls is closer to 14 points (on the candidate’s margin) than 6 or 7 points, it might make them more forgiving on the polls if that candidate loses.
Who actually answers unidentified numbers on their mobile phones anymore? I've asked several people (warning: small sample size!) and none of them do. I certainly don't. Even if the number pulls up a name, if I don't recognize it, I don't bother....they can leave a message if it's my doctor calling to remind me of an appointment.
So, even leaving partisanship aside, what kind of person will pick up and answer personal opinion questions? What are their motivations? Maybe pollsters should be asking more background questions about who these people actually are before they get to the election questions. And then run their profiles against social media, etc. databases to confirm who they really are before they decide how to weight their answers.
Although, if I did happen to accidentally listen to such a session once, I'd for certain never answer an unknown number again!
I don't think we have anyone in the mainstream media, talking about polling the way it should covered. This means that the inaccurate conventional wisdom takes on polling remain.
In 2012, the conventional wisdom was that polls were imperfect, we shouldn't trust them. The polls were pretty accurate and Obama got reelected fairly easily. The general reaction was that we should trust polls.
In 2016, the conventional wisdom was that polls were perfect, but a normal polling error occurred and Trump got elected. The general reaction was that we shouldn't trust polls.
In 2020, the reaction to the polls being off again, have led to reaction that we should never trust polls again.
Maybe it was just people's bias talking, but I saw a lot of takes that Biden would win by double digits and win 413 Electoral votes. I personally thought that was a bit on the crazy side. Biden winning by double digits and Texas seemed unlikely given polarization. I think this may have caused the reaction of "we should never trust the polls again". The "blue shift" effect may also have contributed to that narrative as well since Trump was leading in MI, WI, and PA by a lot on Election Night.
Elliot:
I think the last point here is a very good one. Many of the pieces about polls and forecast were written on election night when the incomplete returns looked much different (redder) than the full returns, but takes have not been rewritten. I would like to see everyone who wrote about the polls from Nov 3rd through 7th reevaluate their takes.
Hello Elliott, If the margin of error generally reported only reflects sampling error and the “true-er” margin of error is really double or more than the sampling error that means typical state polling error is 6 to 10 to maybe 13 percent. It would seem then that polls would then be “reliable” only in states that knowledgeable politicians would be able to call without polling. Meaning it is likely that Utah goes Republican or DC goes Democratic. In states that are in the Lean Dem, Lean Rep or Tossup states to use Cook Political report categories polls provide little directional insight. Do I have that right?
Fred:
It depends on what you mean by “reliable.” Remember that errors at the tail end of the margin of error are still rarer than errors nearer to the center (0), so if a candidate is up 5 in a poll it’s still likelier that they win than that they lose. But if we tell people that the margin of error on any of those polls is closer to 14 points (on the candidate’s margin) than 6 or 7 points, it might make them more forgiving on the polls if that candidate loses.
Here's a question that gets to the sample:
Who actually answers unidentified numbers on their mobile phones anymore? I've asked several people (warning: small sample size!) and none of them do. I certainly don't. Even if the number pulls up a name, if I don't recognize it, I don't bother....they can leave a message if it's my doctor calling to remind me of an appointment.
So, even leaving partisanship aside, what kind of person will pick up and answer personal opinion questions? What are their motivations? Maybe pollsters should be asking more background questions about who these people actually are before they get to the election questions. And then run their profiles against social media, etc. databases to confirm who they really are before they decide how to weight their answers.
Although, if I did happen to accidentally listen to such a session once, I'd for certain never answer an unknown number again!