Optimizing Student Participation in and Benefit From Brief, Digital Learning Skills Training

• In Event: Supporting Undergraduate STEM Students' Academic Success Through Campus-Based and Researcher-Developed Self-Regulated Learning Initiatives

Abstract

Background. Learners benefit from training programs that teach learning strategies and methods of self-regulated learning (Dignath et al., 2008; Hattie, 2008). These trainings can be provided in digital contexts (Broadbent & Poon, 2015), but typically classroom- and technology-delivered programs require hours, days, or evens weeks of training to produce achievement effects (Zheng, 2016). More recent efforts to deliver brief (i.e., 90-120 minutes), digital training within the digital platforms where students learn have produced benefits for learners’ behaviors and achievement (Bernacki et al., 2020a; Bernacki et al., 2020b). However, students’ relatively low completion rates for even these brief, digital trainings suggest further time reduction is needed to broaden the impact of learning skills training.

Aim. We adapted a reading intensive, 90-120 minute Science of Learning to Learn digital skill training program into a shorter, 15-minute format, retaining core topics and interactive components (per Hattie & Donoghue, 2016), to test whether the abbreviated design would produce similar effects on exam scores, at superior completion rates, and thus achieve broadened participation and a strengthened intent to treat effect on those assigned to the training condition.

Methods. Of 496 students enrolled in the introductory biology course, the 283 who consented to participate and share data were randomly assigned to complete a Week 3 “Exam-Check In” activity that asked a few questions about students’ approach to learning and exam preparation (Control Activity); the Treatment Condition also completed the Science of Learning to Learn – Abbreviated Form activity in the middle of the Exam Check-in (i.e., Figure 1; condition blinded to instructors). Students took a first unit exam in Week 5 when face-to-face, had two unit exams disrupted by COVID outbreak and a course migration, and a cumulative final exam after a transition to fully online instruction and assessment.

Results. Task completion (65%) exceeded prior rates (36%, 36%, 45%) and required 13 minutes. A repeated measures ANCOVA controlling for pretest knowledge indicated a significant benefit of the abbreviated treatment, F (1, 157) = 4.212, p = .042, comprising mean differences on the subsequent exam (Exam 1; EMMTREATMENT= 71.8, SD = 15.9 vs. EMMCONTROL= 67.3, SD = 16.3, d = .28) and the cumulative Final Exam (EMMTREATMENT= 71.2, SD = 15.9 vs. EMMCONTROL= 68.4, SD = 15.4, d = .29).

Significance. Students completed an abbreviated version of an established digital skill training program at a higher rate and obtained only slightly lower effects on exam achievement (past ds~.35) than obtained after completing the original version. In a semester disrupted by the outbreak of the COVID pandemic, the abbreviated treatment obtained an immediate effect on exam performance under face-to-face conditions, as well as a delayed effect on summative exam performance after a transition to online instruction. Additional analyses will document how students in the Treatment group responded to training activities as potential moderator, and how training affected students’ learning behaviors (i.e., use of digital content) before and after disruption, self-reported self-efficacy, task value, achievement goals, self-regulation, and coping, as well as feelings of depletion after COVID disruption occurred.

Authors

• Matthew L. Bernacki, University of North Carolina Chapel Hill

• MeganClaire Cogliano, University of Nevada - Las Vegas

• Jenifer Utz, University of Nevada - Las Vegas

• Christy Strong, University of Nevada - Las Vegas

• Kathryn Rafferty, University of Nevada - Las Vegas

• Jonathan C. Hilpert, University of Nevada Las Vegas