Detecting Social Problems From User-Generated Content: A Methodological Reflection

• In Event: Methodological Challenges of Hateful and Problematic Content Detection Online

Abstract

Social problems may be understood as situations problematized by significant groups of population which may be visible in popular discourse developing in social media. Detecting which topics are perceived as problematic by large numbers of internet users may help to prevent social unrest. In this talk we consider how problematized topics can be detected and differentiated from non-problematic topics using a combination of topic modelling (TM) with other techniques. We draw our examples from two studies that exemplify different strategies of social problem detection in Russian online media. The first with detects audience’s problematizations of online news agendas with TM-based sentiment analysis of readers’ comments; the second seeks to mine problematized topics from millions of short informal user messages with semi-supervised topic modeling. We reveal research designs of both studies, as well as challenges that we encountered and possible ways to overcome them.

Author

• Oleg Nagornyy, National Research University Higher School of Economics