The Analysis of Factorial Survey Data: The Utility of Random Coefficient Regression

• In Event: Young Scholars: Undergraduate Student Research

Abstract

Researchers frequently use factorial surveys to measure individual attitudes and decisions, due to these surveys’ unique ability to incorporate the benefits of both randomized experiments and survey methods. In a factorial survey design, respondents read hypothetical scenarios—often called vignettes—in which phrases randomly vary from one respondent to the next. Respondents then answer questions relating to the scenarios seen. By design, when respondents read multiple scenarios, the vignette dimensions are clustered within respondents. As a result, residuals are not independent and their variance tends to be heteroscedastic. Random coefficient (RC) regression models provide accurate coefficient estimates and standard errors when data are clustered, and therefore have been recommended for analyzing factorial survey data. Analyzing factorial survey data using ordinary least squares (OLS) regression, however, may yield similar results to an analysis using an RC model given certain conditions (e.g., a small number of respondents). We perform Monte Carlo simulations to identify factorial survey conditions in which OLS regression produces biased estimates compared to RC regression. Implications and recommendations for researchers will be noted and discussed.

Authors

• Matthew P West, University of Nevada, Las Vegas

• Melissa L Rorie, University of Nevada, Las Vegas