The role of relevance determinations in multiple text reading and writing: An investigation of the MD-TRACE

• In Event: Synergy of Reading and Writing Within Multiple Source Tasks: Novel Approaches and Innovative Techniques

Abstract

Objective
In this investigation, we used meta-data to examine undergraduates’ decision making when selecting and navigating across multiple documents and the relation between those decisions and written response quality.

Theoretical Framework
Information-rich digital environments present students with a multitude of text sources. Learning from multiple texts requires students to continually decide which texts to access, in what order, and for how long, and to regulate these actions to meet task demands. Despite its importance, regulation of information accessed from multiple documents has infrequently been examined. In this study, guided by the Integrated Framework for Multiple Texts (IF-MT; List & Alexander, 2019), we analyzed students’ decision-making in text selection and navigation across multiple documents for evidence of their regulatory behaviors (execution phase). We then related those data to the quality of students’ written reports on the urban housing crisis (production phase).

Method
Undergraduates (n=129) were randomly assigned to compose a research report either about the causes of or the solutions to the US urban housing crisis. Students were presented with 12 digital texts. Half addressed the causes and half presented solutions to the housing crisis. Thus, six of the 12 texts were relevant to each task condition (cause or solution). Constructing the library in this way allowed us to examine the extent to which students monitored relevance during text access. A variety of log-data indicators were collected including: (a) specific texts accessed, (b) order of access, and (c) text processing time.

Results
Cluster analysis was conducted to identify patterns in students’ regulatory processing. Total number of texts accessed, and the percentage of relevant texts students selected, based on the assigned task, were used to form clusters. Four navigation profiles were identified. (Silhouette measure of cohesion: 0.6, indicating good fit.)
Students were classified as knowledge seekers if they accessed a high number of both relevant and irrelevant texts in equal proportion. Students were identified as goal-directed if they accessed relatively few texts and only those relevant to their assigned task. Students were classified as adopting an exploratory path to navigation if they selected a moderate number of texts, which were both task-relevant and task-irrelevant. Finally, students were classified as non-strategic if they accessed relatively few texts irrelevant to their assigned condition. (See Table 1 and Figure 1).
As validation of the resulting clusters, we found that students assigned to different navigation profiles differed significantly in the total time devoted to text access [F(3, 125)=11.60, p<.001, η2=.22], and the quality of written reports [F(3, 120)=4.00, p<.01, η2=.09].

Scholarly Significance
This study contributes significantly to the extant literature on multiple text use in its innovative use of log-data to reveal students’ regulatory processes as they selected and navigated texts, and in its use of cluster analysis to examine patterns in the number and character of the texts chosen. The resulting profiles were not only consistent with those predicted by List and Alexander (2017) but also differentiated among those who generated higher quality reports and those who did not.

Authors

• Hye Yeon Lee, The Pennsylvania State University

• Alexandra List, The Pennsylvania State University