New Data, Old Practices: Perceptions of Public Opinion in the News

• In Event: Truth Hurts: New Data, Old Problems in Political Communication Research

Abstract

Journalists often report on public opinion, and reporting on polls is widespread in news (e.g. Rosenstiel, 2005; Toff, 2016). Journalists have also begun to report on public opinion using a different data source: social media (Anstead & O’Loughlin, 2015; Lukito & Wells, 2018; McGregor, 2019). We aim to determine to what extent journalists’ use of social media data in news reports impact people’s perceptions of public opinion as well as how credible people judge that content to be. These new data sources communicate a strong social message about public opinion, yet one that may further undermine the public’s shaky trust in the news media as an important democratic institution.
Research has focused on how different types of public opinion reported in the press are received by the public. Studies have focused on how survey data and exemplars, or quotes from the public, influence people’s opinions and perceptions of public opinion. Reading a news story with survey data itself has a strong impact on people’s assessments of general public opinion (e.g. Zerback, Koch & Kramer, 2015). Studies find that exemplars have strong direct effects on perceived opinion climate (Daschmann, 2000; Gaskins, Barbaras & Jerit, 2019) as well as indirect effects through their impact on people’s personal opinions (Zerback, Koch & Kramer, 2015). We expect that perceptions of public opinion towards an issue will follow the opinions of the majority in the survey (H1) as well as the opinions of exemplars towards that issue (H2) as reported in a news story. We also expect that stories that combine survey data and exemplar quotes will have a greater impact on perceptions of public opinion, as compared to survey data alone (H3). Alongside traditional survey data and exemplar quotes, journalists now present social media metrics and individual social media posts as forms of public opinion. Much like polls, journalists use social media metrics in service of horserace coverage (Anstead & O’Loughlin, 2015; McGregor, 2019). Journalists also use individual social media posts – mostly from Twitter – as a new form of vox pop or exemplar opinions (Anstead & O’Loughlin, 2015; Lukito & Wells, 2018; McGregor, 2019; Tworek, 2018). Though we know little about how these particular types of public opinion data may impact perceived public opinion, we can draw parallels to the effects of survey data on social media metrics and exemplars on individual social media posts. We expect that perceptions of public opinion towards an issue will follow the opinions of the social media metrics (H4) as well as the opinions of social media exemplars towards that issue (H5) as reported in a news story. We also ask: Is the combined effect of social media metrics and social media exemplars on perceived public opinion greater than the impact of social media metrics alone (RQ1)?
Another important factor in how people interpret various types of public opinion data presented in news is credibility. Given the introduction of social media data and posts into the arena of public opinion data, it’s important to assess how individuals confer credibility to different types of information. We expect that stories that present quantitative information will have greater credibility than those that present qualitative examples (H6). Because social media data is a newer – and generally untrusted (Dubois et al., 2018) – source of public opinion data, we would expect that traditional reports will have greater credibility than social media data (H7). We also expect that stories that combine quantitative information with qualitative examples would have greater credibility than those that only present one type of evidence (H8). Finally, we test the influence of a post-hoc methodological warning message about the limits of social media data in representing public opinion (H9) for those respondents who viewed the social media data conditions.

To test these expectations, we conducted a national, online survey experiment of American adults, drawn from a Qualtrics panel (N=1,100), which was fielded in early January 2020 (the survey is currently in the field). We examine the differences between six experimental conditions (poll only, individual quotes only, Twitter trend metric only, individual tweets only, poll AND individual quotes, Twitter trend metric AND individual tweets), as hypothesized above, on two different public opinion issues, one polarized and one non-polarized. This allows us to control for potential biased processing of public opinion evidence based on respondents prior issue positions and examine how those biases might interact with the processing of diverse public opinion data.

Authors

• Shannon C McGregor, University of North Carolina

• Ozan Kuru, University of Pennsylvania