Multimodal Data Analysis of Students' Self-Regulated Learning Behaviors in Open-Ended Learning Environments

• In Event: 35.031 - Multimodal Data During Learning With Advanced Learning Technologies: What Does Evidence Reveal About Self-Regulated Learning?

Abstract

Interest in developing self-regulated learners using advanced learning technologies led to developing open-ended learning environments (OELEs). In OELEs, students learn and refresh their domain knowledge using hypermedia resources. While these environments encourage exploration, strategic thinking, and developing monitoring skills, K-12 students often have difficulties making progress toward their learning and problem solving goals (Kinnebrew, et al., 2017). The need for engaged concentration can result in cycling through emotional states, such as delight, confusion, frustration, and boredom (Azevedo, et al., 2013; Baker, et al., 2010). Therefore, the ability to track and understand learners’ performance and cognitive, affective, metacognitive, and motivational (CAMM) processes is important for developing OELEs with effective scaffolds that support students.

We focus on Betty’s Brain (Leelawong & Biswas, 2008), an OELE where students learn by building causal maps. From classroom studies, we derive empirical evidence of students’ cognition, metacognition, and affect and how they are influenced by performance and interactions with Betty, the teachable agent, and Mr. Davis, the mentor agent (Munshi, et al., 2018). We also probe relations between students’ eye-gaze patterns when reading and their ability to build causal maps (Rajendran, et al., 2018). These findings provide us with information on how students’ regulatory skills unfold in OELEs, allowing us to consider how adaptive scaffolding can better support learning and performance.

We analyzed students’ logged actions (reading, note-taking, causal-map building, assessing map correctness, and conversations with the mentor agent) and their observed affective states while they worked with Betty’s Brain. Affective state data were collected using the Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP; Ocumpaugh, et al., 2015). Trained observers coded affective states (bored, confused, delight, engaged concentration, frustrated, other) and behavioral categories (on-task, off-task, on-task conversation, other). To study reading behaviors, we aligned the time-stamped log data with gaze data. Students’ eye movements were measured by fixations (duration of gaze focused on the same point) and saccades (movement of gaze between two fixations) (Rayner, 2009).

Results revealed high performers showed more delight, whereas low performers were bored for significantly more time. Low performers showed significantly more boredom when they received feedback from the mentor agent and during reading. High performers showed significantly more delight when assessing the correctness of their map, and when they received feedback from the mentor agent. The mentor agent’s hints and reports on progress led to different cognitive and affective experiences in high versus low performers, which supports previous research on learners’ beliefs about the nature of learning (Dweck, 2002). With the eye-gaze data, we predicted students’ performance in map building (F1 score=0.82).

Our empirical analyses of multi-modal learner data provide evidence for enhancing self-regulated learning theories, which view learning as an active process involving goal-setting, planning, monitoring, and refinement (Zimmerman & Schunk, 2012). Moreover, we look upon self-regulation as a dynamic and evolving process that includes CAMM processes. Our analyses (e.g., Kinnebrew, et al., 2014, Munshi, et al., 2018) establish the interactions that influence learning and performance. In the future, we hope develop causal models of the interaction processes.

Authors

• Gautam Biswas, Vanderbilt University

• Ramkumar Rajendran, Vanderbilt University

• Anabil Munshi, Vanderbilt University