Generalizability in Experimental Political Science

Session Submission Type: Full Paper Panel

Session Description

Identifying externally valid causal effects is a central objective of political scientists across all subfields. Doing so entails theoretical and empirical challenges, which have recently received growing attention, especially among experimental political scientists. Our panel brings together formal theorists, methodologists, and applied scholars to explore different dimensions of generalizability in experimental political science. Egami and Da In Lee will present a paper titled “How to Design Randomized Experiments for External Validity.“ This paper proposes a framework for optimal experimental site selection, laying out practical guidelines for designing randomized experiments by considering internal and external validity. Huang will present a paper titled “Partial Identification of Externally Valid Causal Effects in the Presence of Overlap Violations.” In this paper, they introduce a framework for considering external validity in the presence of overlap violations allowing researchers to partially identify the population average treatment effect and consider the plausibility of the overlap violations. Slough and Tyson will present a paper titled “The Policymaker's Problem” in which they formalize projective validity as one form of external validity and embed projective validity into a decision-theoretic model of a policymaker's use of evidence to set policy. Finally, Bassan-Nygate, Renshon, Weeks, and Weiss will present a paper titled “The Generalizability of IR experiments Beyond the U.S.,” in which they report a harmonized replication study, focusing on four prominent IR experiments implemented in seven different countries – Brazil, Germany, India, Israel, Japan, Nigeria, and USA to examine whether IR experiments that test some of the field’s more well-known theories generalize beyond the U.S. This panel will be chaired by Dustin Tingley and include Anna Wilke and Graeme Blair as discussants. By combing papers that focus on theory, methods, and application, we hope our panel will generate interesting and important conversations regarding the generalizability of experimental evidence in political science.

Sub Unit

• Division 51: Experimental Research

Chair

• Dustin Halliday Tingley Harvard University

Discussants

• Anna Maria Wilke New York University

• Graeme Blair UCLA

Individual Presentations

• Partial Identification of Externally Valid Effects with Overlap Violations - Melody Huang

• How to Design Randomized Experiments for External Validity - Naoki Egami, Columbia University; Diana Da In Lee, Princeton University

• The Policymaker's Problem - Tara L Slough, New York University; Scott Tyson, University of Rochester

• The Generalizability of IR Experiments beyond the U.S. - Lotem Bassan-Nygate, Harvard University; Jonathan Renshon, University of Wisconsin, Madison; Jessica L. P. Weeks, University of Wisconsin-Madison; Chagai Weiss, Stanford University