Model Credibility in Political Science

• In Event: Lightning Round: Research in Five: Graduate Students I

Abstract

Quantitative research in political science investigates social, economic, and political processes by modeling the relationship among factors statistically. While we might not know the true data generating process (DGP), we can use what is known about a set of factors and an outcome of interest to approximate their relationship. Due to the observational nature of a large amount of research in political science, however, the true DGP is unknown - introducing uncertainty at the level of the model. Supplementary analyses that support the credibility of a primary analysis have thus become commonplace. In this paper, I review common methods of assessing model credibility in current-day political science research and build on work that calls for systematic testing of models in the social sciences (Athey and Imbens 2017, Oster 2019, Cinelli and Hazlett 2020). I argue that purposive procedure selection allows researchers to convey information through a selected number of checks that touch upon different dimensions of model uncertainty. I conclude by comparing procedures using simulated and replication data. Selected tests in this paper will be offered in easily accessible, free online software for public use.

Author

• Shana Scogin, University of Notre Dame