Embedding Regression

• In Event: Computational Social Science: New Approaches

Abstract

Political scientists commonly seek to make statements about how a word's usage and meaning varies over contexts---whether that be time, partisan identity, or some other document-level covariate. A promising avenue is "word embeddings" that are specific to a domain, and that simultaneously allow for statements of uncertainty and statistical inference. We introduce the "a la Carte on Text embedding regression model" (ConText regression model) for this exact purpose. In particular, we extend and validate a simple model-based method of "retrofitting" pre-trained embeddings to local contexts that requires minimal input data and out-performs well-known competitors for studying changes in meaning across groups and times. Our approach allows us to speak descriptively of "effects" of covariates on the way that words are understood, and to comment on whether a particular use is statistically significantly different to another. We provide experimental and observational evidence of performance of the model, along with open-source software.

Authors

• Pedro L. Rodriguez, Vanderbilt University

• Arthur Spirling, New York University

• Brandon Michael Stewart, Princeton University