Political Sectarianism during the US 2020 Election: Evidence from Twitter Users

• In Event: Innovative Methods for Studying Political Communication

Abstract

Through growing ideological differences between members of different political parties and the changing nature of partisan information intake, political sectarianism, or the "tendency to adopt a moralized identification with one political group and against another," has surged in the last decade. Expressed through aversion, othering, and moralization towards the members of the opposing political party, this non-ideological type of polarization is hypothesized to pose an existential threat to a healthy democracy (Finkel et al., 2020). Despite political sectarianism's important political consequence, empirical study of this political behavior is often constrained by the lack of measurement tools and data availability.

This paper fills this gap in the literature by developing a transformers-based multi-label deep neural network learning model that allows researchers to identify both affective polarization and political sectarianism on social media. Specifically, we utilize RoBERTa (Liu et al., 2019; Terechshenko et al., 2021). The model features several improvements over standard uses of pretrained language models, including utilizing user profile information and locations as additional text features in the classifier pipeline and using deep ensemble methods (Lakshminarayanan, Pritzel, & Blundell, 2017) to improve classification outcomes.

The training data used with the model is formed from tweets that came between March and December 2020. These tweets were filtered by specific partisan keywords: "Democrat", "Republican", "GOP", "liberal", and "conservative". A team of research assistants labeled 12,374 tweets. Specifically, tweets were labeled for affection and aversion towards Democrats and Republicans, targets of these tweets (general/unspecified, candidates or elected officials, voters, media, and others), the reasoning for the affect, whether the tweet targeted a person or policy, and whether the tweet contained othering and/or moralizing language. Our findings with the labeled dataset indicate that the number of tweets expressing partisan aversion overwhelms the number of tweets expressing affection, suggesting the prevalence of affective polarization on Twitter in 2020. Tweets expressing moralization and othering take up a smaller but nontrivial proportion, a large number of which specifically target politicians. In terms of intercoder reliability, coders agreed most highly on the aversion categories, and struggled the most to agree on the othering and moralization categories. As a result, performance, in terms of area under the precision-recall curve, is robust for the categories of aversion towards Republicans or Democrats, and weaker for the othering and moralization categories. Including user profile information and locations as additional text features further improves classification outcomes; deep ensemble methods slightly improve classification outcomes as well. Current work is being done to expand the training set beyond the five keywords and to define the moralization and othering categories more narrowly in an attempt to improve the performance of the classifier in these areas.

This paper contributes to text-as-data methods for social media opinion measures, as it is one of the first general-purpose political sectarianism classifiers. Substantively, it will empower further investigation of the dynamic of affective polarization and political sectarianism on social media.

Authors

• Haohan Chen, New York University

• Zhanna Terechshenko, New York University

• Patrick Y. Wu, New York University

• Jonathan Nagler, New York University

• Richard Bonneau, NYU

• Joshua A. Tucker, New York University