Measuring Self-Regulated Learning With a Multi-Agent Hypermedia Environment

• In Event: Measuring Self-Regulated Learning With Multi-Agent Learning Environments

Abstract

Our paper will focus on the use of various methods to collect and understand the complex nature of cognitive, metacognitive, and affective processes during learning with MetaTutor, a multi-agent learning environment for human biology. Current methodological approaches to studying SRL processes have several weaknesses and therefore limit our ability to advance theoretical, methodological, and analytical issues. Therefore, our approach has been to use MetaTutor as an innovative technology-rich assessment tool with which to collect trace data of cognitive, metacognitive, and affective processes during learning. Theoretically, we integrate several cognitive and socio-cognitive models of SRL (i.e., Winne, Hadwin, Zimmerman, Schunk, and Pekrun). By integrating these models of SRL, we are able to use MetaTutor’s technology rich assessments to measure and better understand what students know, how they feel and why they behave in so many different ways.

We will report the results of an experimental, multi-day study involving 120 university students who used MetaTutor to learn about the human circulatory system. They were instructed to use several key SRL processes during their learning session (e.g. assess their emerging understanding). Participants were randomly assigned to one of three conditions—control, prompt, or prompt and feedback. In the prompt and feedback condition, students received timely prompts delivered by the embedded pedagogical agents to use the SRL processes and then received feedback regarding the accuracy or effectiveness of their use of these processes. Those in the prompt condition received the same prompts but not the feedback. Lastly, those in the control condition did not receive any prompt or feedback from the agents. During the two-hour learning session with MetaTutor, we collected concurrent think-alouds, eye-tracking, video recording of the face (for affect detection and classification), text log files (quiz results, summaries and metacognitive judgments) and notes and drawings. We also collected pretest and posttest data and several self-report measures on agent likeability and metacognitive knowledge about specific SRL processes.

We will present several quantitative and qualitative results stemming from the various data sources. In particular, we will focus on the learning efficiency and learning time, significant fluctuations in affect during the learning sessions, eye-tracking processes which reveal participants’ selection, organization, and integration of multiple representations of information (which are indicative of cognitive and metacognitive processes), and log-file data which details the duration and sequencing of specific behaviors (e.g. navigational profiles), learners’ accuracy in making metacognitive judgments during learning, emotion regulation during different phases of learning, and qualitative changes in students’ self-regulatory processes based on learner-agents dialogue moves. The results will demonstrate the advantage of using multi-agent environments to advance our understanding of what, how, when, and why students’ know and feel during complex learning. The data sources will provide evidence that has the potential to advance current conceptual, theoretical, methodological, and analytical frameworks related to SRL processes. These advances will in turn allow researchers to design more effective multi-agent learning environments that are sensitive and responsive to students’ cognitive, metacognitive, and affective needs during learning.

Authors

• Roger Azevedo, McGill University

• Reza Feyzi Behnagh, McGill University

• Jason Matthew Harley, McGill University

• François Bouchet, McGill University

• Gregory Trevors, McGill University

• Melissa Duffy, McGill University

• Zaynab Sabagh, McGill University