Measurement Directiveness as a Cause of Response Bias: Evidence from Three Survey Experiments

• In Event: 361 - Section on Methodology Paper Session. Open Topic I

Abstract

Extant research comparing survey self-reports of normative behavior to direct observations and time diary data have yielded evidence of extensive measurement bias. However, most of this research program has relied on observational data, comparing independent samples from the same target population, rather than comparing survey self-reports to a criterion measure for individual respondents. The present research addresses this next step using data from three studies. In each study, respondents completed a conventional survey questionnaire, including questions about frequency of religious behavior. Respondents were then asked to participate in a text messaging (SMS) data collection procedure, reporting either (1) participation in religious behavior specifically or (2) all changes in major activity without explicitly specifying religious behavior. Findings suggest that directive measurement, priming the respondent to consider the focal behavior, is a cause of measurement bias.

Authors

• Philip S. Brenner, University of Massachusetts-Boston

• Shane Sharp, Northern Illinois University

• John D. DeLamater, University of Wisconsin-Madison