Generalizability of Digital Learning Behavior Predictions Across University Courses: An Empirical Study

• In Event: Feature Engineering for Learning Analytics in Higher Education Research on Motivation and Self-Regulation

Abstract

STEM dropout rates at German universities are alarmingly high; in computer science, the rate is 44% (Heublein et al., 2022). Many students leave due to performance difficulties, especially in theory-heavy courses like formal logic (Soll et al., 2023). Digital learning environments have emerged to support students’ learning processes (Geck et al., 2019), and digital learning behavior predicts student performance (Deininger et al., 2023; K. Schneider et al., 2019). Besides survey data, clickstream data provides crucial insights into students' learning processes. However, the generalizability of such data beyond specific courses and universities remains under-researched. Furthermore, it is unclear if early- semester data suffice for timely predictions and interventions (cf. Cogliano et al., 2022).
We investigate whether data from a different university course can be used for out-of-sample predictions, verifying model generalizability beyond the original context. We also identify which learning behavior variables are key predictors. Lastly, we progressively omit data from the semester's end to its start to determine how early we can predict student success.

Authors

• Jakob Schwerter, University of Tübingen

• Judith Bose, TU Dortmund University

• Philipp Doebler, TU Dortmund University

• Marko Schmellenkamp, RUB Bochum

• Thomas Zeume, RUB Bochum