Lurking Heterogeneity and Theoretical Inference in Political Science

• In Event: Causal Inference: Accounting For Heterogeneity

Abstract

Heterogeneous treatment effects have long concerned both methodologists and empiricists, as increased variance makes it more difficult to reject a null hypothesis all else equal. We propose a new perspective: identifying heterogeneity in effects can reveal important insights about political and social phenomena that are not apparent from studying average treatment effects. In this paper, we provide a unified framework for understanding how the presence of heterogeneous effects can alter theoretical inferences in dramatic ways. We refer to failures to address this problem as "lurking heterogeneity." We demonstrate how lurking heterogeneity generates mistaken or incomplete theoretical inferences in three different areas of research from across political science. We then provide guidelines for researchers to both avoid this inference problem (e.g., in experimental design) and evaluate whether theoretical inferences in their own work are sensitive to lurking heterogeneity (e.g., in observational studies). Our guidelines thus parallel and build upon insights from more qualitative traditions that insist upon intersectional or holistic analysis, while offering concrete procedures that scholars using quantitative methods can undertake.

Authors

• Rachel Bernhard, University of California, Davis

• Ryan Copus, University of Missouri - Kansas City School of Law

• Ryan Hubert, University of California, Davis