a few years ago I worked on an ML powered application that was distributing tv ad money across networks and programs. TV ads dont have the benefit of clickthrough data. instead, we used correlations like: people who watch Friends buy Nissan SUVs and shop at natural markets. they tend to go on roadtrips instead of taking the plane.. and the like. I'd try to use to correlate voter data with lifestyle preferences and from there figure out which slices are missing from poll samples.
It has become more common in recent years for pollsters to match their data to lists of registered voters that also have consumer information on many individuals. I haven't heard anything about using these data to create non-response weights, though. The information tends to be very noisy, and it's unclear to me what benchmark you would use for, eg, the share of Americans that own a Ford F-150.
Have folks tried to do backwards looking polls to try and quantify the selection bias? One could imagine a very simple experiment where you take a poll of known voters during '16 (so that you don't have to deal with flawed turnout estimates) and try and recover the precinct level or county level vote shares. From there you should be able to get a nonresponse estimate, assuming you mitigate partisan social desirability bias (using randomization or some other technique).
Some pollsters perform a similar adjustment in their polls where they weight their data to be representative of the previous results of the presidential election. This year proved to be an important test of this exercise, as the pollsters that did this still overestimated support for Biden. That suggests that non-response may not have been limited to the traditional D / R / I splits, but happening even within the categories, with Republican respondents who filled out the survey being more pro-Biden than the overall group. If that's true, it's going to be a tough problem for pollsters to solve.
a few years ago I worked on an ML powered application that was distributing tv ad money across networks and programs. TV ads dont have the benefit of clickthrough data. instead, we used correlations like: people who watch Friends buy Nissan SUVs and shop at natural markets. they tend to go on roadtrips instead of taking the plane.. and the like. I'd try to use to correlate voter data with lifestyle preferences and from there figure out which slices are missing from poll samples.
Marya:
It has become more common in recent years for pollsters to match their data to lists of registered voters that also have consumer information on many individuals. I haven't heard anything about using these data to create non-response weights, though. The information tends to be very noisy, and it's unclear to me what benchmark you would use for, eg, the share of Americans that own a Ford F-150.
Have folks tried to do backwards looking polls to try and quantify the selection bias? One could imagine a very simple experiment where you take a poll of known voters during '16 (so that you don't have to deal with flawed turnout estimates) and try and recover the precinct level or county level vote shares. From there you should be able to get a nonresponse estimate, assuming you mitigate partisan social desirability bias (using randomization or some other technique).
Juan:
Some pollsters perform a similar adjustment in their polls where they weight their data to be representative of the previous results of the presidential election. This year proved to be an important test of this exercise, as the pollsters that did this still overestimated support for Biden. That suggests that non-response may not have been limited to the traditional D / R / I splits, but happening even within the categories, with Republican respondents who filled out the survey being more pro-Biden than the overall group. If that's true, it's going to be a tough problem for pollsters to solve.