Generalized Kernel Regularized Least Squares

• In Event: Poster Session: Political Methodology & Political Science Education & Civic Engagement
  In Poster Session: Poster Session: Political Methodology

Abstract

Kernel regularized least squares (KRLS) is a popular method that allows for flexible estimation of non-linear effects and interactions between variables. However, existing implementations are mostly limited to linear models and cannot incorporate substantively important extensions such as non-interacted variables (e.g. treatment indicators) or random effects to account for heterogeneity.

We introduce generalized KRLS (gKRLS) to address these limitations by first recasting the KRLS problem as a carefully specified hierarchical model. Doing so allows a number of benefits; first, it allows for easy estimation of the strength of regularization in non-linear models without requiring cross-validation. Second, it is straightforward to include extensions such as random effects or interactions between a treatment and the kernel. Third, estimation can leverage existing, highly-optimized, software that already commonly used by applied researchers. We further enable scalability to potentially huge datasets by implementing random sketching.

We find that when compared against traditional KRLS, gKRLS performs nearly as well (and sometimes better) in the traditional linear model and that random sketching incurs a relatively modest cost in terms of performance at a large gain in terms of speed. We then show that using gKRLS provides important benefits in more complex problems (e.g., non-linear models with necessary random effects) that are missed by ignoring those features of the model.

Authors

• Qing Chang

• Max Goplerud, University of Pittsburgh