Methodological Challenges for Detecting Incivility on Social Media

• In Event: Methodological Challenges of Hateful and Problematic Content Detection Online

Abstract

Incivility seems an innate component of social interaction online (Masannari, 2014; Phillips, 2015). Research has examined the nature and causes of uncivil discourses online (Hmielowski, Hutchens & Ciccirillo, 2014; Rowe, 2014), and the effects it has on perceptions of the discussion and future interactions (Gervais, 2015; Muddiman & Stroud, 2017). Online sites moderate uncivil discourse by keyword blocks and users’ reporting. Due to the multi-dimensional nature of incivility (Stryker, Conway & Danielson, 2016), these approaches are not able to mitigate incivility. We have used supervised machine learning to identify uncivil comments generated by the public on Facebook and Twitter during the U.S. 2016 presidential campaign. We review current efforts to detect incivility, describe our approaches to classify incivility, and detail challenges at developing valid algorithms. We reflect on the limits of algorithmic detection of incivility and the need for researchers to know their data in order to develop accurate models.

Gervais, B. T. (2015). Incivility online: Affective and behavioral reactions to uncivil political posts in a Web-based experiment. Journal of Information Technology & Politics, 12, 167-185.

Hmielowski, J. D., Hutchens, M. J., & Ciccirillo, V. J. (2014). Living in an age of online incivility: Examining the conditional indirect effects of online discussion on political flaming. Information, Communication & Society, 17, 1196-1211.
Massanari, A. (2015). #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technoculutres. New Media & Society, 19, 329-346.
Phillips, W. (2015). This is why we can’t have nice things: Mapping the relationship between online trolling and mainstream culture. Cambridge, MA: MIT Press.
Rowe, I. (2014). Civility 2.0: A comparative analysis of incivility in online political discussion. Information, Communication & Society, 17, 121-138.
Stryker, R., Conway, B. A., & Danielson, J. T. (2016). What is political incivility. Communication Monographs, 83, 535-556.

Authors

• Jennifer Stromer-Galley, Syracuse U

• Patricia G. C. Rossini, School of Information Studies, Syracuse U

• Feifei Zhang, Syracuse U

• Erin Bartolo, Syracuse U