Shedding the Shirkers: Methods for De-Randomizing Survey Responses

• In Event: Crowdsourcing

Abstract

The explosion of online survey providers has raised concerns about respondent shirking: respondents who pay little attention to their response or even answer at random. We put forward a design- and model-based solution to detect and correct for survey shirkers. Our design-based solution employs special attention checks and weighting to identify and correct for insincere survey respondents. Our model-based solution identifies which survey questions and respondents are most prone to generating insincere responses and uses this information to estimate genuine survey responses levels. We provide the conditions for our model based approach to successfully detect shirkers in past surveys and show in simulations that the model is effective at identifying random responses. We apply our method to estimate how shirking affects support for political violence against out-partisans in the US.

Authors

• Matthew Tyler, Stanford University

• Justin Grimmer, Stanford University