Do Indicators of Source Quality Affect News Diets? Evidence from an Experiment

• In Event: News Engagement

Abstract

As the primary arena for viral misinformation shifts toward transnational threats such as the Covid-19 pandemic, the search continues for scalable, lasting countermeasures compatible with principles of transparency and free expression. Solutions to date have tended to focus on improving people’s ability to recognize quality news, but to our knowledge few if any interventions have demonstrably reduced real-world exposure to online misinformation. We test if this can be done using live informational feedback of source reliability information to online users. We find that this intervention can dramatically reduce real-world exposure to misinformation as well as reduce belief in popular misinformation and improve trust in authoritative media sources. Using data from a pre-registered field experiment conducted in the summer of 2020 we assess the effectiveness of NewsGuard, a prominent web browser extension that displays independently produced source reliability information of online news domains to users. A user encounters reliability ratings of news sites in two distinct instances. A reliability rating of a news site appears on the user’s toolbar when they visit a news site. It also appears alongside hyperlinks in their search engine results and social media feeds. We conducted a two-wave panel survey of respondents that included an encouragement to install NewsGuard for three weeks. This two-wave online panel survey was fielded by the survey company YouGov in the summer of 2020 (wave 1, May 28 to June 9, 2020, N=3,862; wave 2, June 19, 2020 to July 1, 2020, N=3,337). A subset of these respondents were members of the YouGov Pulse panel and voluntarily provided behavioral data on their online information consumption (N=939). 1,998 respondents were encouraged to install NewsGuard and 1,339 respondents served as the control group. Using data from two survey waves (one survey at the beginning of the treatment period and a second survey post-treatment) and web-tracking data from a subset of respondents, we find that installing the extension for three weeks decreased the proportion of unreliable news sites respondents visited by 33% and reduced the perceived veracity of popular misinformation spread about the Black Lives Matter movement and Covid-19. Unlike other strategies to counteract misperceptions, our intervention seems to achieve these effects without reducing the perceived veracity of true information. Finally, installing this browser extension increased trust in the mainstream media — raising the possibility of a self-sustaining cycle of credible news consumption. These encouraging results indicate that unobtrusive, adaptive informational feedback can persistently increase the quality of people’s news diets, reduce belief in misinformation, and improve trust in authoritative media sources. Our study joins a small but growing literature examining the effectiveness of online tools and nudges using randomized encouragement designs, precise measures of treatment compliance, and digital trace data.

Authors

• Kevin Aslett, New York University

• Andy Guess, Princeton University

• Joshua A. Tucker, New York University

• Richard Bonneau, NYU

• Jonathan Nagler, New York University