Conservative Talk Radio and Misinformation

• In Event: The Conservative Media Ecosystem

Abstract

The 2016 US election brought a lot of interest in studying misinformation. There has been a growing body of work on estimating the prevalence of misinformation (e.g., Guess et al., 2019) and designing interventions to correct misbeliefs (e.g., Porter & Wood, 2021). The overwhelming focus of the emerging field of misinformation studies is social media. Although there have been attempts to evaluate misinformation at the scale of the information ecosystem (See Allen et al., 2020), no studies have systematically examined the prevalence of misinformation in one of the most popular media formats in the US, namely political talk radio.
According to Pew Research, 83% of Americans listened to terrestrial radio in a given week. As a popular genre of radio, political talk radio is popular among Republican voters – especially older voters living in rural areas. According to Talkers magazine, top nationally syndicated political talk radio shows can often reach millions of listeners. For example, Sean Hannity’s weekday talk radio show boasts an audience size of 16.25 million. These hosts are adept at building emotional bonds with their listeners (Berry & Sobieraj, 2013; Rosenwald, 2019). Studies showed that political talk radio can significantly shape listeners’ belief and behavior (Barker, 2002).
There is evidence suggesting that conservative talk radio shows may contain a high volume of misinformation and conspiracy theories. For example, in the 1990s Rush Limbaugh promoted the conspiracy theory that Bill Clinton and Hillary Clinton were implicated in the death of Vince Foster, a former White House counsel; in 2009, he spread the birther conspiracy theory, which falsely claimed that Barack Obama was not born in the United States.
The reason why political talk radio has eluded most misinformation/disinformation scholars is perhaps a methodological one. Compared to text-based social media data, audio data from talk radio are harder to collect and analyze. In this paper, we seek to fill this gap by providing an estimate of the prevalence of misinformation about voter fraud in the 2020 US election – based on the largest political talk radio data ever collected.
Through web scraping, we collected approximately 2600-hour audio data of the top 9 (by audience size) nationally syndicated conservative political talk radio shows between October 1, 2020 and March 31, 2021, including Sean Hannity, Rush Limbaugh (until Feb.2 2021), Dan Bongino, Mark Levin, and others. Our analytic strategies consist of three steps. First, we leverage existing audio features such as length of pauses, background music, and embedded commercials to divide each audio file into 5-10 min segments. The goal of segmentation is to facilitate manual coding. Second, we build an online annotation interface, which allows annotators to listen to a given audio segment and read the transcripts as they annotate the presence of misinformation. Lastly, we use annotated audio files as training data to help design an AI-empowered system that uses both verbal and nonverbal cues to monitor political talk radio for the presence of voter fraud related misinformation in future elections.
This paper is part of a larger inter-disciplinary effort aimed at helping fact-checkers detect misinformation in political talk radio. We believe that our project will make a significant contribution to the field of misinformation and disinformation. Theoretically, it will yield new insights about the quality of information consumed by an understudied population (older, rural, and Republican). Methodologically, it will demonstrate how advanced methods in NLP combined with human input can be used to help detect political misinformation in audio data at scale.
























References:


Allen, Jennifer, et al. "Evaluating the fake news problem at the scale of the information ecosystem." Science advances 6.14 (2020): eaay3539.

Barker, D. (2002). Rushed to judgment: Talk radio, persuasion, and American political behavior. Columbia University Press.

Berry, J. M., & Sobieraj, S. (2013). The outrage industry: Political opinion media and the new incivility. Oxford University Press.

Guess, A., Nagler, J., & Tucker, J. (2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science advances, 5(1), eaau4586.

Porter, E., & Wood, T. J. (2021). The global effectiveness of fact-checking: Evidence from simultaneous experiments in Argentina, Nigeria, South Africa, and the United Kingdom. Proceedings of the National Academy of Sciences, 118(37), e2104235118.

Rosenwald, B. (2019). Talk radio’s America: How an industry took over a political party that took over the United States. Harvard University Press.

Authors

• Ruihong Huang

• Lu Tang, Texas A&M University

• Yunkang Yang