Partial Identification of Externally Valid Effects with Overlap Violations

• In Event: Generalizability in Experimental Political Science

Abstract

Estimating externally valid causal effects is a foundational problem in the social sciences. A key assumption in generalizing or transporting causal effects is overlap. However, in practice, having full overlap between an experimental sample and a target population can be challenging, due to feasibility constraints, ethical concerns, or contextual differences between the experiment and the population. In the following paper, we introduce a framework for considering external validity in the presence of overlap violations. The contributions are two-fold. First, the framework allows researchers to partially identify the population average treatment effect under overlap violations. Furthermore, we introduce a method for researchers to perform valid inference over such a set. Second, we introduce summary measures and benchmarking that help researchers consider the plausibility of the overlap violations. Importantly, the paper provides a mathematical framework for researchers to consider different sampling and experimental designs when designing externally valid experiments. The proposed partial identification approach is illustrated on a set of deep canvassing experiments.

Author

• Melody Huang