How Well Do Young People Deal With Contradictory and Unreliable Information Online? What the PISA (Programme for International Student Assessment) Digital Reading Assessment Tells Us

• In Event: Learning From Text in Today’s Society: Reconciling Multiple Incoherent Sources of Information

Abstract

With the advent of the internet, infinite quantities of information have become available to almost
everyone, and an ever-increasing proportion of reading, especially by younger people, takes place in
digital environments. This entails new demands on readers. The traditional mechanisms in print
publishing that exert some control over the reliability of information are largely absent in the online
environment. Operating successfully in the digital medium requires not only access to technology, but
also the ability to integrate, evaluate and communicate information (Warschauer, 1999). Faced with large
amounts of information and limited time, readers must make immediate evaluations of the usefulness of
different sources, in terms not only of relevance but also of trustworthiness. Readers now need to make
their own choices about which information to read, and which to trust.
There is sometimes an assumption that young people, as “digital natives”, are able to use online
information effectively, including selecting and negotiating digital texts that are not only relevant for
what they need, but also are likely to provide reliable information. This paper examines the question of
how well young people are in fact able to recognise whether information is likely to be trustworthy, in the
context of the Digital Reading Assessment (DRA) that was part of the Organisation for Economic Cooperation
and Development’s Programme for International Student Assessment (OECD PISA) in 2009.
While some small-scale work has been done in this area (for example, Leu & Castek, 2006), PISA 2009
offers the first large-scale international assessment of digital reading. The DRA was administered to
around 25,000 students, in 19 countries and 15 languages. The assessment included several tasks with a
focus on the evaluation of texts for reliability and trustworthiness, representing an explicit strand in the
PISA reading framework (OECD, 2010).
This paper will examine in detail some tasks from the PISA DRA, and investigate students’ success in
completing evaluative tasks on line. It will relate these tasks to descriptions of developing proficiency in
this important aspect of 21st century reading.
Results showed that tasks demanding the critical appraisal of texts for credibility or trustworthiness are
high on the reading proficiency scale: consistently, such tasks are completed successfully by fewer than
30% of students. Increasing proficiency in evaluating texts is described in the levels of the empiricallyderived
proficiency scale for digital reading (OECD, 2011).
There is an assumption that students are able to use online resources that are not only relevant to the tasks
they are set, but also are likely to provide trustworthy information. In order to do this, students need to
have criteria for evaluating information. This study provides empirical evidence that the great majority of
15-year-old students lack the ability to make basic evaluations about the credibility and trustworthiness of
digital texts. Students need to be taught how to make these evaluations. This message is of critical
importance to policy makers and teachers.

Authors

• Tom Lumley, Australian Council for Educational Research

• Juliette F. Mendelovits, Australian Council for Educational Research