Supervised Classification of Political Texts Using Multilingual Embeddings

• In Event: Embeddings in Multiple Languages

Abstract

Large portions of political text collections are written in multiple languages and principally invite comparative quantitative analysis. However, established approaches to automated analysis of text in multiple languages require either linguistically qualified human coders, human translators, or high-quality machine translation and thus tend to stymie comparative research for both practical and methodological reasons. In this paper, I propose an alternative approach to cross-lingual automated text analysis that relies on multilingual text embedding methods to enable language-independent supervised text classification: Texts written in different languages are embedded in a joint semantic space using a publicly available multilingual language model. The resulting text embeddings are then used as features to train a supervised machine learning classifier. To validate the proposed approach, I conduct a series of text classification experiments on three different political text corpora. These experiments show that classifiers trained on multilingual text embeddings pass three important tests: They classify held-out documents as accurately as comparable classifiers trained on monolingual or translated texts; they perform by and large consistently across languages; and they classify documents written in languages that were not present in the training data with little to no loss in predictive performance, thus enabling cross-lingual classification. Viewed together, these results present supervised classification from multilingual text embeddings as a reliable, replicable, and cost-efficient approach to multilingual text classification in political science research. This study thus contributes to an emerging methodological literature on multilingual quantitative text analysis in political science. To ease the application of the proposed method and to encourage further research, I open-source a multilingual text embedding R package developed for this study.

Author

• Hauke Licht, University of Cologne, Cologne Center for Comparative Politics