A Gaussian Process Framework for Social Science Models

• In Event: Better Models, Better Predictions

Abstract

Social scientists are typically interested in learning substantively motivated quantities such as marginal effects, first differences, or treatment effects in settings where they are hard to isolate accurately from background factors such as clustering, temporal autocorrelation, or spatial confounding. Accurate estimation requires imposing substantively motivated assumptions, typically implemented as part of a linear model. Examples include unit fixed effects or specifying spatio-temporal autocorrelation structures. More flexible modeling approaches such as random forests, neural networks, or generalized additive models, allow researchers to relax strict linearity assumptions, but are typically ill-suited in these more complex settings since imposing substantively-motivated assumptions on the data-generating process is difficult or impossible to implement without building custom-made software.

In the paper, we introduce the powerful Gaussian process (GP) framework for modeling social science data which offers a superior compromise between the restrictive assumptions of most linear models and the highly agnostic assumptions behind the most common machine learning methods in the literature. We begin by introducing the GP framework and approach to inference, noting that many linear models already in the literature are actually special (and restrictive) cases within the GP framework. We then illustrate how to leverage the power of GPs to build far more flexible models in the presence of clustering and spatio-temporal autocorrelation. We demonstrate how to fit GPs modularly using the GPyTorch framework, focusing on both estimation and interpreting key quantities of interest to social scientists. We conclude by discussing further possible extensions of GPs to other common settings in the social sciences.

Authors

• Yehu Chen, Washington University in St. Louis

• Roman Garnett, Washington University in St. Louis

• T. Ryan Johnson, Washington University in St. Louis

• Jacob M. Montgomery, Washington University in St. Louis

• Annamaria Prati, Washington University in St. Louis