Not a Black Box: Tools for Interpreting Deep Neural Language Models

• In Event: Applications in Political Methodology Using Large Language Models

Abstract

As computational political science research adapts an increasingly data-driven approach (Grimmer, Roberts and Stewart 2022), the ability to characterize the behavior of complex models is of ever greater importance in order to better identify the scope of inductive claims made on the basis of these models. This is particularly the case in political science text-as-data, where pre-trained deep neural language models have become the de facto standard in the broader discipline. To political scientists, these models offer opportunities to study hard-to-detect linguistic phenomena with smaller quantities of training data. Nevertheless, a lack of interpretable parameters remains an obstacle to the critical application of these models in social science research.
In this paper I introduce a collection of tools and approaches from the fields of interpretable machine learning/explainable AI (IML/XAI) and adapt them to the standards of interpretability and validation employed in quantitative political science research. Specifically, I examine feature importance metrics for transformer-based language models.
In the first section of this paper I discuss 'explanation' in quantitative political science research and how this differs to IML/XAI. In the second I define and explore several feature importance metrics applicable to transformer-based language models, and contrast their interpretation with that of regression coefficients in an OLS model. In the third section I present an empirical approach to validating these metrics based on the FRESH framework (Jain et al 2020). In the final section, I present two substantive social science applications for these metrics: characterizing partisan language in parliamentary speech and validating automated hate speech detection models. I provide a software package for the suite of interpretation and validation tools presented in this paper to simplify and encourage their application.

Author

• Musashi Jacobs-Harukawa, Princeton University