Theory-Driven Data Mining to Understand Self-Regulated Learning Processing in a Higher Education Biology Course

• In Event: 51.024 - Educational Data Mining: Affordances and Constraints in the Analysis of Behavior and Performance in STEM

Abstract

Theoretical Framework
Science educators have embraced active learning pedagogies as a way to foster deeper understanding, and reach a broader audience, than more traditional pedagogies have afforded in the past (Haak et al., 2011; PCAST, 2012). These efforts have been successful, particularly for first-generation and other students underrepresented in science fields (Eddy & Hogan, 2014; Freeman et al., 2014). However, students’ likelihood of benefitting from such pedagogies depends upon the degree to which they can successfully self-regulate their learning (Greene, 2018; Sinatra & Taasoobshirazi, 2018). Self-regulated learning (SRL; Zimmerman, 2013) requires effortfully enacting the knowledge, skills, and dispositions necessary to make plans, set goals, monitor performance, vary learning strategies, and productively reflect upon both the process and products of learning. SRL performance is a strong predictor of academic success in science (Dent & Koenka, 2016; Eilam & Reiter, 2014), and more research is needed to identify and assist those students who might struggle to self-regulate in active learning courses (Ben-Eliyahu & Bernacki, 2015).

Method
In our study, 408 college students enrolled in an active learning, introductory biology course utilized a learning management system (LMS) for a variety of important course activities including taking quizzes, making appointments to seek help from the instructor, and downloading materials for studying and reflecting upon learning performance. We leveraged the vast array of students’ LMS data to make inferences about their ability to effectively enact SRL and predict subsequent performance on course assessments. The LMS data were numerate, fine-grained, and temporal; we relied on theory to identify and group student activities into a reasonable number of interpretable classes of SRL (i.e., metacognition, cognitive help-seeking, course management; Bernacki, 2018), which we could analyze using differential sequence mining (DSM; Kinnebrew, Loretz, & Biswas, 2013). Our research question was: Are there differences in the frequency of SRL processing between students who scored less than 70% on course exams and those who scored higher?

Results
Between these groups, we did not find differentially frequent activity patterns during the period prior to the first exam. However, in every subsequent exam period (i.e., between one exam and the next), we found that students who scored lower than 70% more frequently submitted their homework close to the deadline, compared to their peers who scored higher on the exam. We interpreted this as evidence of procrastination, a common indicator of problematic SRL processing (Wolters, Won, & Hussain, 2017). Students scoring 70% or higher on the exam more frequently engaged with the LMS, including logging in, downloading course materials, and reviewing exams.

Significance
In this study, DSM analyses revealed varied patterns of engagement predictive of exam performance, but only after we engaged in theory-driven identification and aggregation of the fine-grained LMS data. Our methods and findings demonstrate the important role of theory in shaping the LMS data, and how to use those data to identify struggling science students early in their career when there is time to offer remediation and support. DSM provided an effective means of mining the vast amount of LMS data.

Authors

• Jeff A. Greene, University of North Carolina - Chapel Hill

• Christopher J. Urban, University of North Carolina - Chapel Hill

• Robert D Plumley, University of North Carolina - Chapel Hill

• Matthew L. Bernacki, University of North Carolina Chapel Hill

• Kathleen M. Gates, University of North Carolina - Chapel Hill

• Kelly Hogan, University of North Carolina - Chapel Hill

• Cynthia Demetriou, University of North Carolina- Chapel Hill

• A. T. Panter, University of North Carolina - Chapel Hill