Understanding Self-Regulatory Processes Using Multimodal Trace Data During Human-Machine Interactions With an Intelligent Tutoring System

• In Event: 35.031 - Multimodal Data During Learning With Advanced Learning Technologies: What Does Evidence Reveal About Self-Regulated Learning?

Abstract

New methods for detecting, tracking, measuring, and analyzing multimodal self-regulated learning (SRL) data that have specific non-static attributes (i.e., frequency of use, time-dependent patterns of use), offer novel ways to examine and understand the role of these processes across advanced learning technologies (ALTs), contexts, age groups, etc. (Azevedo et al., 2018). These methods can reveal patterns of SRL events, based on the use of various types of multimodal data (e.g., log files, eye tracking, facial expressions of emotion, physiological sensors) that can significantly enhance our current understanding of the sequential and temporal nature of SRL (Biswas et al., 2018; Taub et al., 2017; Winne, 2018). The focus of our presentation is to present the issues associated with capturing, analyzing, and inferring CAMM SRL processes from multimodal data during learning with an ALT.

Data on 170 instrumented undergraduates participating in a multi-day experiment with MetaTutor to learn about the circulatory system will be used to exemplify key issues related to multimodal SRL data (see Figure 1). Participants were randomly assigned to the adaptive or non-adaptive condition. In the adaptive condition, participants were prompted to use several key SRL processes during learning (e.g., activating relevant prior knowledge, using effective learning strategies) by the agents embedded in MetaTutor. During the 2-hour learning session, we collected the following process data from each participant: eye tracking, video recording of the face, log files, notes, and physiological data (see Figure 2).

Our results focus on describing the strengths and weaknesses of using multimodal multichannel SRL process data to advance our conceptual, theoretical, methodological, analytical, and educational challenges. For example, micro-level data provide information on: (1) fluctuations in affective states (e.g., frequencies of learning-centered emotions), (2) eye-tracking processes (e.g., gaze behaviors on specific areas of interest [AOIs]), and (3) log-file data, which details the duration and sequencing of specific behaviors (e.g., frequency and time spent on relevant content). Mid-level data (1) represents learners’ accuracy in making metacognitive judgments (related to calibration and overconfidence in mastery of multimedia content related to a particular learning goal); (2) provides information on the deployment of cognitive and metacognitive processes based on the frequency of using the SRL palette; (3) illustrates their emotion generation and regulation during different sub-goals; (4) provides information on their regulatory processes associated with adaptive changes during the learning session; (5) reveals their knowledge integration across representations of information; (6) exemplifies changes in their self-regulatory processes based on learner-agent dialogue moves, feedback, and scaffolding; and, (7) provides evidence of how the deployment of SRL processes is associated with knowledge construction activities (e.g., taking notes) and is predictive of overall performance.

The data sources and analyses presented in this presentation provide evidence that has the potential to advance: (1) conceptual and theoretical issues, including the conceptualization of SRL vs. other-RL; and (2) an interdisciplinary research agenda and analytical framework for measuring, analyzing and inferring multichannel CAMM SRL processes from multimodal data that is applicable for designing intelligent, CAMM-sensitive ALTs.

Authors

• Roger Azevedo, University of Central Florida

• Michelle Taub, University of Central Florida

• Elizabeth Brooke Cloude, University of Central Florida

• Megan Price, University of Central Florida