A Self-Regulated Learning Analytics Prediction-and-Intervention Design: Detecting and Supporting Struggling Biology Students

• In Event: Supporting Undergraduate STEM Students' Academic Success Through Campus-Based and Researcher-Developed Self-Regulated Learning Initiatives

Abstract

Background. As undergraduate STEM courses undergo curricular reform, instructional designs are becoming richer, and students engage with more digital resources for active learning (Olson & Riordan, 2012; Eddy & Hogan, 2014). However, as digital learning environments become more complex, students who lack the skills to navigate these environments and work productively with these resources can languish; such students need to acquire the capacity to self-regulate their learning in complex tasks that characterize STEM degree programs (Greene, 2018).

Aim & Methods. We investigated the effects of a learning analytics-driven prediction modeling platform and a brief digital self-regulated learning skill training program targeted to support undergraduate biology students identified as likely to fail the course. Using learning analytics methods, a prediction model comprising prior knowledge scores and learning management system log data of student activities during the first two weeks in the course was applied to flag students who were likely to receive a C or worse (N = 143). Students who were flagged were randomized into a Flagged Treatment (N = 79) or Flagged Control (N = 64) condition (see Figure 1). The primary goals of this work were to examine whether training self-regulated learning strategies (i.e., retrieval practice, distributed practice, self-explanation) improved students’ achievement in the course, and whether such achievement benefits led to a greater proportion of students progressing towards their science degrees.

Results. Providing training to students who were flagged as likely to perform poorly significantly improved their achievement on unit exams, compared to students who were also flagged as likely to perform poorly but did not receive the training (Flagged Control). The effect of training on final examination was mediated by unit exam achievement (see Figure 2). In addition, the students who were predicted to perform well (N = 83; students predicted to receive a B or better) and flagged treatment groups did not differ statistically significantly on academic performance. Training also had a significant effect on final course performance with students in the flagged treatment and non-flagged groups outperforming the flagged control students (see Figure 3).

Significance. The results indicate that an algorithm that uses behavioral data to predict achievement does so with sufficient accuracy to detect the large differences in achievement earned by two groups of learners distinguishable by their early, digital learning behaviors, and that a brief ~15-minute digital skills training was sufficient to ameliorate these achievement differences when deployed before the first unit exam is administered. When the prediction and intervention findings are considered through the lens of self-regulated learning theory, the behaviors that informed the prediction model can be interpreted as indicators of cognitive and metacognitive processes conducted using learning resources designed to support task engagement (Roll & Winne, 2015, Winne, 2017).The effects obtained by training also confirmed that a corresponding training grounded to cognitive strategies to be enacted and metacognitive processes enacted to monitor and control learning overcame these initial differences between students who do not exhibit self-regulatory online behaviors early in a course and those who appear to do so.

Authors

• MeganClaire Cogliano, University of Nevada - Las Vegas

• Matthew L. Bernacki, University of North Carolina Chapel Hill

• Jonathan C. Hilpert, University of Nevada Las Vegas