Using Online Measures to Understand Self-Regulated Learning With Advanced Learning Technologies

• In Event: Integrating Different Approaches to Investigating Self-Regulated Learning

Abstract

Self-regulated learning (SRL) with advanced learning technologies involves a complex cycle of temporally unfolding cognitive and metacognitive processes that impact students’ learning. We present issues related to treating SRL as an event as well as the strengths and challenges of using online trace methodologies to detect, trace, model, and foster students’ SRL processes. We provide our theoretically driven assumptions regarding the use of several cognitive methodologies, including concurrent think aloud protocols, log-files, and eye-tracking, and provide several examples of empirical evidence regarding the advantages of treating SRL as an event. Last, we discuss challenges for measuring cognitive and metacognitive processes in the context of MetaTutor, an intelligent adaptive hypermedia learning environment.

Most models of SRL propose a general time-ordered sequence that students follow as they perform a task, but there is no strong assumption that the phases are hierarchically or linearly structured. Our assumptions are in line with Winne and Hadwin’s (2008) theory of SRL. Based on our extensive use of on-line methods, we further assume that: (1) it is possible to detect, trace, model, and foster SRL processes during learning; (2) that understanding the complex, dynamic nature of the unfolding regulatory processes during learning with technology environments is critical in determining why certain processes are used; (3) that the use of SRL processes can dynamically change over time and that the unfolding of SRL is cyclical in nature; and, (4) that capturing, identifying, and classifying SRL processes used during learning with hypermedia environments are challenging tasks.

Data for this study comes from 90 undergraduates who used MetaTutor to learn about the circulatory system during a two-hour experiment. Despite using three experimental conditions, this submission will only focus on the 30 participants in the control condition, so as to examine students’ deployment of SRL without the SRL scaffolding that artificial pedagogical agents provide in the other conditions. Though several on-line data sources were collected during the experiment, we will focus exclusively on the results of the concurrent think-aloud, log-file, and eye-tracking data.

We will provide a synthesis of the results, emphasizing issues and insights related to the strengths and weaknesses of collecting, coding, analyzing, and interpreting process data. One insight we will discuss is the importance of the classification of these processes at various levels of granularity and valence. We will also discuss issues related to the temporal alignment of several data streams (e.g., concurrent think-alouds with eye-tracking data), which are key to understanding the unfolding of the processes in real time and providing evidence of behavioral signatures associated with specific SRL processes. For example, some on-line measures need to be augmented with other measures and methods in order to provide converging evidence. The use of log-file data to generate hypotheses regarding fundamental assumptions about SRL (e.g., agency, adaptations) will also be proposed. Lastly, we will discuss and present evidence regarding how on-line measures can be converged with other process, product, and self-report data to provide a comprehensive understanding of SRL measurement during learning with advanced learning technologies.

Authors

• Roger Azevedo, McGill University

• Jason Matthew Harley, McGill University

• Reza Feyzi Behnagh, McGill University

• François Bouchet, McGill University