Typology of External Validity: A Causal Inference Perspective

• In Event: Causal Inference: Design and Generalization

Abstract

External validity of randomized experiments has been a focus of long-standing methodological debates in the social sciences. However, the multi-dimensionality of the external validity problem is often overlooked. The formal definitions of the external validity have been incomplete, with researchers typically focusing only on one or two aspects of the problem. In this article, we provide a typology of external validity using a causal inference framework. First, building on an influential work by Shadish et al. (2002), we decompose the external validity into four types; (1) X-validity, whether units are representative of the target population, (2) Y-validity, whether the outcomes measured in experiments are comparable to the target outcome of interest, (3) T-validity, whether treatments randomized in experiments are comparable to the target treatment of interest, and (4) C-validity, whether contexts of experiments are comparable to the target context of interest. We then present assumptions required to estimate the population average treatment effect (PATE), each assumption corresponding to one of the four validity types above. Finally, we provide a general estimator for the PATE and illustrate our proposed methodology using simulation studies and the reanalysis of a GOTV experiment.

Authors

• Naoki Egami, Columbia University

• Erin Hartman, UC Berkeley