The Distribution of Hate Speech and Its Implications for Content Moderation

• In Event: Negative Side-Effects of Social Media: Extremism & Hate Speech

Abstract

Hate speech in online spaces is widely perceived as one of the main obstacles to constructive debates, especially because it targets marginalized groups disproportionately. Consequently, policy-makers and activist groups have put forward strategies to counter hate speech, and researchers have begun to examine the effectiveness of different approaches. However, a greater understanding of the specific nature of online hate speech would empower the search for potential responses to curb it. This paper contributes to this question in two main ways, relying on an original dataset of over 23 million tweets posted in 2021 by over 77,000 unique users interested in Swiss politics, and using a custom-built multilingual classifier to identify hate speech. First, by consistently tracking the online behavior of a fixed set of users, we are able to evaluate the frequency of hate speech relative to other user generated content. Our approach allows us to establish that hate speech is relatively rare (less than 1% of content, on average) and that more than 50% of hate speech is produced by 1% of users. Second, the paper examines the effect of different counterspeech strategies using a field experiment conducted on Twitter. In contrast to previous studies, we find that counterspeech strategies do not, on average, significantly reduce hate speech. Against the background that only few users are responsible for most hate speech, our experimental results suggest that strong haters may be more difficult to persuade, and that responses to hate speech may need to differentiate among different kinds of hate speech producers. To the extent that most hate speech is produced by a small subset of high-volume users, addressing the problem may require more invasive forms of content moderation than counterspeech. Understanding which types of interventions work in which context and for which users will be an important task for future research.

Authors

• Dominik Hangartner, ETH Zurich

• Gloria Gennaro, UCL

• Laura Bronner, Department of Government, LSE

• Maël Dominique Kubli, University of Zurich

• Laurenz Derksen, ETH Zürich

• Fabrizio Gilardi, University of Zurich

• Karsten Donnay, University of Zurich