How to Design Randomized Experiments for External Validity

• In Event: Generalizability in Experimental Political Science

Abstract

Over the last few decades, social scientists have used randomized controlled trials and quasi-experimental methods to make valid causal inferences. This so-called credibility revolution has almost exclusively focused on internal validity and paid little attention to the fundamental issue of external validity — how scientists and policymakers can generalize causal findings beyond a specific study. To address this concern, most existing approaches aim to improve the external validity of randomized experiments that are designed without external validity in mind. However, as the general principle of causal inference suggests, it is best to change the experimental design rather than change how to analyze experiments after data collection. In this project, we propose a framework and practical guideline for designing randomized experiments by considering both internal and external validity. In particular, we propose a novel approach to selecting experimental sites optimally, while allowing researchers to incorporate various logistical and ethical constraints.

Authors

• Naoki Egami, Columbia University

• Diana Da In Lee, Princeton University