Image for Wary Conservatives, Eager Activists Combine to Produce Poll Bias

Political polls are pilloried for routinely understating the views of conservative Trump voters. One reason is the reluctance of those conservative voters to participate in political polls. Ironically, a left-leaning organization is trying to improve the odds.

Data for Progress, in a retrospective of 2020 polling, has identified “response bias” as the culprit for off-base polls. “Conservative white voters are opting out of polling, while liberal voters are disproportionately opting in, creating an underlying bias in our respondent pools. We also have evidence that liberal partisan activists are systematically overrepresented in our surveys,” according to the report.

The group’s goal is to find ways, including unconventional techniques such as email and text message outreach, to win the confidence of reluctant Republican respondents to make poll samples more representative. Simultaneously, it also tries to downgrade over-represented liberal activists who eagerly participate in public opinion surveys.

Under-representation of conservative voters in polls has been blamed for the inaccurate projection Hillary Clinton would win the presidency in 2016. The “insight” pollsters drew from that miscue was to distinguish between white voters with and without college degrees, which has become a standard feature of political polling. The Data for Progress team isn’t satisfied with that distinction. They have found conservatives with college degrees aren’t always keen on participating in surveys either.

The team believes the most telling indicator is where voters live. A person’s zip code, they say, is a better clue to a point of view or political preference than educational levels. If those zip codes are rural areas under-represented in a poll sample, then the poll results are likely to be skewed. “A college-educated white Republican in a rural, consistently Trump-supporting ZIP code is different from one in an urban or Biden-supporting ZIP code,” Johannes Fischer, the lead methodologist at Data for Progress, told The New York Times. “Not all white conservative voters with college degrees will vote similarly – and not all of them will be equally likely to respond to a survey.

Pollsters for Date for Progress have experimented with the language they use to invite survey participation. “Can we use language that elicits more authoritarian responses, or keys into language used more often on the right, to engage those voters?” Fischer explains. The result has been a noticeable increase in participation.

A college-educated white Republican in a rural, consistently Trump-supporting ZIP code is different from one in an urban or Biden-supporting ZIP code.

The skew can be worse, Fischer adds, if the Biden-zip codes in a polling sample are over-represented by political liberal activists. The team is looking for ways to “down-weight” people they identify as activists, who are eager to participate, but don’t always represent a mainstream perspective.

In analyzing the 2020 president election, the group found Joe Biden gained more votes than Democratic House candidates in heavily white suburban and urban counties, while underperforming Democratic House candidates in majority non-white counties. It also found that voter education levels weren’t a totally accurate indicator of voting for Biden and Trump.

“Our research surfaced dozens of zip code characteristics that correlate with support for Trump in 2016 and 2020. For example, the percentage of college educated people in a person’s zip code was a stronger predictor of Biden support than a person’s individual education level,” the report said.

“While these findings might seem counter-intuitive, political science literature shows that the strongest Democratic and Democratically trending areas are characterized by higher population density, higher concentrations of college-educated people, more diversity and growing economies that are based on professional and waged service work,” the report concluded. “In contrast, the areas that are the most dedicated to Trump typically have a lower population density, higher concentrations of noncollege whites and have economies that depend more on goods-producing sectors, often in declining post-industrial regions.” 

Going forward, Data for Progress says, “We’re applying findings about the importance of local geography to a zip code weighting scheme, which combines characteristics of respondents with the characteristics of their local geography. In fact, retroactively applying these weights to our 2020 pre-election surveys, we see that our new weighting techniques would have significantly decreased polling error.”