A Corrective Methodology to Classifying Misinformation on Social Media

• In Event: Misinformed? Who, How, and Why

Abstract

Misinformation (i.e., information that is inaccurate or misleading) has become a popular research topic within political science and scholars increasingly use computational methods to estimate its scope and dynamics. In 2018, Vosoughi et al. published a seminal paper investigating the spread of true and false news online, which has become a cornerstone in political science scholarship on misinformation. While the paper is admirable and has many key insights, it also suffers from a problematic classification of misinformation. While the methodology is also popularly adopted by many political scientists, this paper will argue that it leads to an overestimation of the overall amount of misinformation.

To challenge the methodology, this paper develops a different approach to identify misinformation, which acknowledges that what is often labeled misinformation is in fact countering misinformation using irony or sarcasm. Similar to Vosoughi et al. and most studies of online misinformation, our methodology is based on verified false stories. But our method departs from the dominant approach, also used by Vosoughi et al. who use hyperlinks to identify tweets related to fact-checking sites, whereas we extract keywords from the verified stories and use human annotators to classify what type of information we are dealing with. This allows us to identify and investigate not just misinformation, but also a range of counter-misinformation, such as irony or negation of false stories.

Our findings are based on 800 million Tweets from the 10th of January 2020 to the 7th of November 2020 and have been collected from Twitter's API using Covid-19 related keywords. We find more information trying to counter-misinformation than actual misinformation and suspect that the countering misinformation is usually counted as misinformation in existing studies. We thus conclude that most existing classification methods may be flawed, leading to an overestimation of misinformation. On those grounds, we propose a new methodology of classifying misinformation on social media, that enables us to identify and possibly target the actual spreaders of misinformation and not just those mentioning misinformation. Lastly, we show that a more thorough understanding of not just misinformation, but also counter-misinformation and the groups that actively fight misinformation is central to understanding the spread of misinformation.

Authors

• Nicklas Johansen, University of Copenhagen

• Rebecca Adler-Nissen, University of Copenhagen