The Effectiveness of Pedagogical Agents' Adaptive Scaffolding on Self-Regulation During Complex Learning With an Intelligent Tutoring System

• In Event: Meaningful Scaffolds to Foster Learning in Interactive Learning Settings

Abstract

Objectives and Theoretical Framework
Improving self-regulation through adaptive scaffolding delivered by pedagogical agents (PAs) during complex learning with intelligent tutoring systems is of critical importance (Azevedo et al., in press). Using Winne and Azevedo (2014) theoretical perspective on self-regulated learning (SRL) and extensive empirical research on scaffolding SRL (Bannert et al., 2015) with learning technologies such as MetaTutor (Azevedo et al., in press) we designed and tested the effectiveness of PAs to scaffold students’ in regulating their own learning about complex science topics while MetaTutor to learn biology. We converged product data with several on-line measures of cognitive, affective, and metacognitive processes (e.g., log files, use of SRL palette, facial expressions of emotions) to examine the effectiveness of scaffolding with (i.e., the adaptive version of MetaTutor) or without the agents (i.e., the non-adaptive version of MetaTutor).

Methods and Results
100 students took part in a 2-day experiment with MetaTutor to learn about the human circulatory system. They were instructed to use several key SRL processes during their learning session (e.g., activating relevant prior knowledge and setting-relevant learning goals, assessing metacognitive processes [e.g., FOK, JOL], effective learning strategies). Participants were randomly assigned either to the adaptive or non-adaptive condition. The effectiveness of PAs’ metacognitive scaffolding was assessed based on analyses of participants’ 2-hour session with MetaTutor where we collected the following data from each participant: eye-tracking, video recording of the face (for affect detection and classification), log-files (e.g., quiz results, metacognitive judgments, learner-agent dialogue), and notes and drawings. We also collected pretest and posttest data and several self-report measures on agent likeability and metacognitive knowledge about specific SRL processes, and motivational traits (e.g., self-efficacy). Our results will focus on describing the effectiveness of PAs’ metacognitive scaffolding by using multi-level trace (process) data, participants’ self-regulatory behaviors, and how they are related to learning outcomes. For example, how on-line metacognitive judgments, log-file data, facial expressions of emotional states, and eye-tracking, relate to underlying metacognitive processes and their relation to learning outcomes.
Results indicated statistically significant differences between MetaTutor conditions, such that learners in the adaptive scaffolding condition had higher learning gains, sub-goal quiz scores, spent more time engaging in each learning sub-goal, engaged in more help-seeking behavior, inspected more relevant and less irrelevant multimedia materials during learning, deployed more sophisticated learning strategies, and made higher metacognitive judgments, compared to participants in the non-adaptive condition. These results suggest the importance of adaptive scaffolding in learning about complex topics and promoting the effective use of SRL strategies while interacting with intelligent tutoing systems.

Scientific and Educational Significance
Understanding the effectiveness of PAs’ scaffolding on SRL is key to enhancing complex STEM learning and performance. The data sources will provide evidence that has the potential to advance current conceptual, theoretical, methodological, and analytical frameworks related to scaffolding and SRL, based on self- and external-regulation (Winne & Azevedo, 2014). These advances will in turn allow researchers to design more effective PAs and multi-agent learning environments that are sensitive and responsive to students’ CAM processes during learning.

Authors

• Roger Azevedo, North Carolina State University

• Michelle Taub, North Carolina State University

• Nicholas Vincent Mudrick, North Carolina State University

• Megan Price, North Carolina State University