Aligning Log-File and Facial Expression Data to Validate Assumptions Linking Self-Regulated Learning, Metacognitive Monitoring, and Emotions During Learning

• In Event: Self-Regulated Learning Analytics: Aligning Data and Their Treatment to the Assumptions of Theory

Abstract

Objectives. The roles of metacognition and emotions during self-regulated learning (SRL) with advanced learning technologies remains elusive. Assumptions underlying models of SRL (Winne & Hadwin, 2008) posit that when learning with hypermedia, metacognitive judgments are likely to be followed by particular affective states (e.g., Azevedo et al., 2013). To test these assumptions, we collected multi-channel trace data (i.e., facial expressions and log files) to examine students’ metacognitive judgments and their subsequent affective states during learning with MetaTutor. This requires extensive data cleaning and aligning of different data channels with different sampling rates (i.e., ms-level, 30 Hz). The goal of this submission is to demonstrate how to align multiple data channels to test if metacognitive judgments (e.g., negative content evaluation; CE-) are followed by particular affective states (e.g., confusion), in line with models of SRL (e.g., D’Mello & Graesser, 2014).

Methods and Preliminary Results. Log-files and emotion expression data were coded from 194 undergraduates while learning with MetaTutor (Azevedo et al., 2013). During learning, they were required to set at least two sub-goals. Data extracted from the log-files included pages students visited, and whether or not those pages were relevant to the active sub-goal. We then calculated ratios: (# of Relevant (or Irrelevant)Pages)/(Total # of Pages), and compared them between conditions (prompt and feedback [P+F] vs. control [C]), and determined that in the P+F condition, students visited significantly more relevant pages; F(1, 191) = 10.56, p < .05, and significantly fewer irrelevant pages; F(1, 191) = 10.60, p < .05, than students in the C condition.
Subsequently we examined differences in students’ affective states when visiting relevant and irrelevant pages by condition. Log-files from a subset of the participants were aligned with their respective facial expression data; i.e., face recordings and numerical output (see Figure 1). The alignment process was conducted within The Observer XT (Noldus Observer XT, 2015) software used to correct the temporal gap between multiple data channels. More specifically, facial expression data captured by Attention Tool (Attention Tool, 2015) is aligned with the text log-file, and the screen recording, by matching the onset of the learning program. This procedure sets all data channels relative to a common starting point. The synchronized recordings can then be viewed alongside the output of coded facial expressions in order to analyze the correspondence of affective states and recorded events (see Figure 2). Next, all data (see Figure 3) can be analyzed to investigate the affective state that occurred at the moment of a metacognitive monitoring process.

Significance. Aligning multi-channel trace data is key in providing empirical evidence to test assumptions from models of SRL. The work reported in this paper is key in advancing models of SRL (Efklides, 2011), and in determining if the affective states proposed to follow metacognitive judgments, such as CE-, do, in fact, follow them. This work has implications for predicting student behavior, and for designing intelligent tutoring systems that can provide individualized instruction including regulating emotions and enhancing the accuracy and quality of metacognitive judgments.

Authors

• Michelle Taub, North Carolina State University

• Roger Azevedo, North Carolina State University

• Seth A. Martin, North Carolina State University

• Garrett C. Millar, North Carolina State University

• Franz Wortha, Technische Universitaet Dresden