Are There Ordering Effects in Conjoint Experiments?

• In Event: New Frontiers in Experimental Design: Conjoint, Field Experiments, and Beyond

Abstract

Conjoint experiments have become a popular method in political science to measure citizens preferences on complex, multidimensional choices or evaluations by requiring respondents to consider trade-offs. This paper investigates whether the order in which attributes are presented in conjoint survey experiments can affect the findings in political science research. To test for potential bias in the ordering of attributes, we have designed a forced choice conjoint experiment. Half of the respondents will receive the attributes in a random order, while the other half will receive the attributes in a fixed order. Value randomization will occur as typically done in conjoint experiments. This experimental design is unique as it isolates the causal effect of having a fixed vs. random order in conjoint experiments, which has not been previously examined in political science literature. The present study will make important contributions to ongoing research on conjoint experiments by providing new insights into the effects of attribute ordering. This novel experiment will help to further the understanding of how to design reliable and valid conjoint experiments.

Authors

• Semra Sevi, University of Toronto

• Md Mujahedul Islam, University of Toronto

• Peter Loewen, University of Toronto

• Sarah Lachance, University of Toronto

• Thomas Bergeron, University of Toronto

• Blake Lee-Whiting, University of Toronto

• Beatrice Magistro, California Institute of Technology

• Clareta Treger, University of Toronto