As a survey researcher, my rule of thumb for a single poll was that expected error was triple the random error, so that 1.96 s.d. was about .6745 s.d., or the reported 95% confidence interval was actually about 50%. At 50%, it' was an even money bet whether results would be within the 95% random confidence interval. For an average of many polls over time, the random component basically disappears (see the Dornsife USC panel poll), leaving at most the expected two-thirds that isn't random. But we don't know much about the distribution of this non-random error. If it followed the normal distribution and paralleled the random error but just didn't average out, then the estimated 95% confidence interval would be about an estimated 80% interval for non-random error, more or less the type of interval you and 538 were estimating. But once we rightly junk the random error basis of calculations, as we do with averages, the few historic episodes of presidential elections with many unknown base distributions for variables provide no defensible grounds for any estimates of reliability. Yes, we want to estimate as a practical matter, but statistics, estimated error, and confidence intervals -- is there really much basis for saying that postwar economic measurements provide some reliable basis for estimating voting error likelihood in a covid-19 economy with stimulus?
There is a lot here. Thanks for your comment. But instead of just answering your questions here, where few people will see my comments back, I’m going to write a post about it later. Maybe Monday.
As a survey researcher, my rule of thumb for a single poll was that expected error was triple the random error, so that 1.96 s.d. was about .6745 s.d., or the reported 95% confidence interval was actually about 50%. At 50%, it' was an even money bet whether results would be within the 95% random confidence interval. For an average of many polls over time, the random component basically disappears (see the Dornsife USC panel poll), leaving at most the expected two-thirds that isn't random. But we don't know much about the distribution of this non-random error. If it followed the normal distribution and paralleled the random error but just didn't average out, then the estimated 95% confidence interval would be about an estimated 80% interval for non-random error, more or less the type of interval you and 538 were estimating. But once we rightly junk the random error basis of calculations, as we do with averages, the few historic episodes of presidential elections with many unknown base distributions for variables provide no defensible grounds for any estimates of reliability. Yes, we want to estimate as a practical matter, but statistics, estimated error, and confidence intervals -- is there really much basis for saying that postwar economic measurements provide some reliable basis for estimating voting error likelihood in a covid-19 economy with stimulus?
Douglas:
There is a lot here. Thanks for your comment. But instead of just answering your questions here, where few people will see my comments back, I’m going to write a post about it later. Maybe Monday.