Automated News: Better Than Expected?

• In Event: Robots, Machines, and Algorithms: Perspectives on Automated Journalism

Abstract

Advances in information technology and natural language generation have made it possible to automatically create written news articles from raw data. We conducted two experiments to study people’s prior expectations and actual perceptions regarding quality, readability, and credibility of such automated in comparison to human-written news. The results were similar across both experiments. First, participants expected more from human-written news in terms of readability and quality but not in terms of credibility. Second, participants’ prior expectations were rarely met for both human-written and automated news. Third, actual perceptions depended on the experimental setting. When participants saw only one article, perceived differences were small. However, when participants saw two articles at once, one each per source, human-written news were rated higher on readability, whereas automated news were favored for credibility. Our results contest previous claims according to which expectation adjustment explains the small differences in perceptions of human-written and automated news.

Authors

• Mario Haim, LMU Munich

• Andreas Graefe, LMU Munich