Confronting the New Gate Keepers: Reassessing Norms in Experimental Research

• In Event: Innovations in Experimental Methods

Abstract

In recent years the subfield of IR has witnessed a rapid increase in the use of experimental methods to test hypotheses about topics in FPA, IS, and IPE. This has allowed scholars to test the micro-foundations of many of the most consequential theories in IR. With this growth has come increased scrutiny. Researchers, reviewers, and journal editors have sought to devise best practices for experimental work. However well-intended, these efforts only serve to increase the cost of experimental research. Without any scientific basis, journals are starting to require pre-registration plans exclusively for experimental research; researchers aggrandize the use of nationally representative samples, even when there is no inferential target; and editors and reviewers often expect multiple experiments in a paper, when only one is necessary for causal inference. These norms are not only theoretically bankrupt, they serve to exacerbate racial, gender, and institutional inequalities in the discipline. We highlight the problems of these norms and provide a set of recommendations to head off these bad practices. In short, researchers should stop propagating these norms in the papers they submit and reviewers and editors should start rejecting papers that do.

Authors

• Danielle Lupton, Colgate University

• Clayton McLaughlin Webb, University of Kansas