Putting Polling Errors into Context

• In Event: Drawing Better Inferences from Biased Surveys

Abstract

Case studies of polling failures have focused on within-election differences in poll accuracy across firms, sponsors, time, geography, and survey methods. The crucial question why the polls failed in a given election but not in others has often remained a matter of speculation. Our aim is to develop a contextual understanding of polling errors and their triggers. We cover a broad range of features of the electoral contest which pundits and scholars have suspected of encouraging polling errors, including candidacies, mobilization, polarization, and electoral conduct. These features vary widely across the 367 U.S. Senate elections that we cover in our empirical study of 9,298 pre-election polls, 1990-2022. We adopt and extend a Bayesian hierarchical modelling approach that allows us to disentangle systematic and random error components at the election level, to extrapolate error to election day, and then to link election features to specific error components.

Authors

• Sina Chen, University of Konstanz

• John Körtner, University of Lausanne

• Peter Selb, University of Konstanz