Estimating Politically Expressive Preference with Context-Aware Named Entities

• In Event: Methodological Developments in Text-as-Data

Abstract

Ideological positions of political parties are commonly measured using text scaling methods, such as Wordfish (Slapin and Proksch 2008) and Wordscore (Laver, Benoit, and Garry 2003; Lowe 2008; Lowe and Benoit 2013). These analysis meth- ods are not only useful for extracting latent dimension of parties and politicians, but also for testing political theories, e.g., intra-party competition (Haber 2015; Däubler and Benoit 2015), and party positionings (e.g. Catalinac 2017; Proksch and Slapin 2010). Concerns have been raised around the performances in internal validity (Bruinsma and Gemenis 2019), and around the assumptions that both Wordscores and Wordfish make of their input data (Egerod and Klemmensen 2020; Proksch and Slapin 2009). In this paper, we propose a novel estimation strategy that augments the text scaling methods used in political science with named entity recognition (NER) that recognizes various types of entity objects. We first extract named entities using state-of-the-art language models in German and English and scale them with Wordfish Model. We illustrate our strategy using the case of parliamentary debates on the topics of immigration issues discussed in the UK House of Commons and the German Bundestag from 2014 to 2019. Party positions are estimated and validated with the Chapel Hill Expert Survey (CHES). To evaluate the performance of extracted entities scaled in our estimation strategy, we cross- validate our method by performing multiple LDA topic models. Then, we shuffle the corpus randomly and split them into k groups by folds. We take one of folds as a test set while using remaining folds as a train set. Afterwards, the average perplexity scores extracted from a series of topic numbers are used to evaluate the quality of estimation between the basic Wordfish model and our approach. We find that (1) parties’ positions estimated by our approach are highly consistent with the CHES expert's judgment, and (2) our approach achieves better internal validity in terms of perplexity performance than the basic text scaling model without the pre-processing step using NER.

References

Bruinsma, Bastiaan, and Kostas Gemenis. 2019. “Validating Wordscores: The Promises and Pitfalls of Computational Text Scaling.” Communication Methods and Measures 13 (3): 212–227.
Catalinac, Amy. 2017. “Positioning under Alternative Electoral Systems: Evidence from Japanese Candidate Election Manifestos.” American Political Science Review 112 (1): 31–48.
Däubler, Thomas, and Kenneth Benoit. 2015. “Scaling Hand-coded Political Texts to Learn More about Left-right Policy Content.” Party Politics 28 (5): 1–11.
Egerod, BC, and Robert Klemmensen. 2020. “Scaling Political Positions from Texts: Assumptions, Methods and Pittfalls.” The SAGE Handbook of Research Methods in Political Science and International relations. Thousand Oaks: Sage, 498–521.
Haber, Matthias. 2015. “The Legislative Consequences of Internal Conflict and Inter- party Divisions.” Research and Politics 2 (2): 298–313.
Laver, Michael, Kenneth Benoit, and John Garry. 2003. “Extracting Policy Positions from Political Texts Using Words as Data.” American Political Science Review 97
(2): 311–331.
Lowe, Will. 2008. “Understanding Wordscores.” Political Analysis 16:356–371.
Lowe, Will, and Kenneth Benoit. 2013. “Validating Estimates of Latent Traits from Textual Data Using Human Judgment as a Benchmark.” Political Analysis 21 (3): 298–313.
Proksch, Sven-Oliver, and Jonathan B. Slapin. 2009. “How to Avoid Pitfalls in Statistical Analysis of Political Texts: The Case of Germany.” German Politics 18 (3): 323–344.
. 2010. “Position Taking in European Parliament Speeches.” British Journal of Political Science 40 (3): 587–611.
Slapin, Jonathan B., and Sven-Oliver Proksch. 2008. “A Scaling Model for Estimating Time-Series Party Positions from Texts.” American Journal of Political Science 52
(3): 705–722.

Authors

• Yen-Chieh Liao, University of Birmingham

• Thomas Saalfeld, University of Bamberg

• David Beck, University of Bamberg