Do Source Evaluations Determine Source Preference? A Mixed-Method Analysis

• In Event: Sourcing: Identifying and Evaluating Information in the Digital Age

Abstract

Theoretical Frame

Using information on the Internet to learn about a topic has been identified as a highly challenging task for learners (Goldman & Scardamalia, 2013). In part, students may be challenged by the volume of information inherently introduced by any task requiring source use online (Lazonder, Biemans, Wopereis, 2000; Thompson, 2003). Among the strategies students have been found to use in culling through online content is to judge sources as useful or not, and, less frequently, as trustworthy or not (McCrudden & Schraw, 2007; Rieh & Belkin, 1998).

A number of studies have examined students’ judgments of trustworthiness and usefulness and found these to differ across document types (e.g., newspaper, textbook; Braasch et al., 2013; Gerjets et al., 2011; Rouet et al., 1996). However, as students increasingly use digital sources (i.e., created online, for digital consumption), like Wikipedia, for Internet research, more work is needed to understand how students evaluate the trustworthiness and usefulness of these sources vis-à-vis traditional source types (i.e., those having print analogues) and balance considerations of trustworthiness and usefulness during source use.

This study examines students’ (a) ratings of trustworthiness and usefulness for digital and traditional source types, (b) justifications offered for ratings along each dimension, and (c) the association between source evaluations and students’ source preferences.

Methods

Undergraduates (n=197) were asked to use a library of six sources, including both digital (e.g., Wikipedia) and traditional (e.g., newspaper) source types, to research a topic and compose a response. Students were asked to rate the usefulness and trustworthiness of each source they accessed and to justify their evaluations. Justifications were coded into six categories, including whether they were based on source features (e.g., author, document type), scope of content, source relevance, or perceived bias.

Time codes were used to determine students’ preferred source (i.e., source they devoted the most time to); usefulness and trustworthiness ratings and evaluation justifications for preferred and non-preferred sources were examined.

Results

Students’ evaluations differed significantly across source types. The newspaper was rated as the most trustworthy (M=71.20, SD=20.37) and the most useful source (M=72.61, 19.48); the blog was scored as the least trustworthy (M=30.17, SD=22.63) and Twitter, the least useful (M=43.29; SD=29.05). Students’ preferred sources included both traditional and digital source types (i.e., newspaper, preferred by: 42.42% of students; analysis essay, preferred by: 26.77%; Wikipedia, preferred by: 18.69%).

Across source types, students who preferred a source rated it as both significantly more trustworthy and significantly more useful than students who used, but did not prefer, that source (e.g., Wikipedia usefulness ratings, preferred: M=71.94, SD=20.44; non-preferred: M=61.65, SD=23.30). Further, justifications for preferred sources were found to reflect different dimensions than justifications for non-preferred sources.

Scholarly Significance

This study examines usefulness and trustworthiness ratings and evaluation justifications to understand how students judged traditional and digital source types. Students’ source preferences were found to reflect both usefulness and trustworthiness considerations, while evaluation justifications were found to function in somewhat distinct ways across source types and in guided students’ source preferences.

Author

• Alexandra List, Ball State University