Leveraging Learning Analytics, Digital Training, and Campus Academic Success Programs to Support to STEM Learners

• In Event: Supporting Undergraduate STEM Students' Academic Success Through Campus-Based and Researcher-Developed Self-Regulated Learning Initiatives

Abstract

Background. Institutions invest considerable resources into learning centers and discipline-specific units that provide academic support to students. Students who take advantage of these services often benefit (e.g., Howlett et al., 2020), but students largely underutilize these services, or begin to use them only after poor performances begin to accrue in critical courses. Institutions have also begun to invest in learning analytics solutions that make use of the data students produce in the earliest weeks of semesters, and apply algorithms that can identify students who are likely to benefit from campus provided support such as academic success coaching and supplemental instruction programs, which often train students’ studying and self-regulated learning skills (Winne & Hadwin, 1998).
Aim. The purpose of this study was to investigate the effects of using a learning analytics-driven prediction model to recommend students complete academic success coaching sessions alongside a digital self-regulated learning training (Bernacki et al., 2020). A second aim was to examine how coaches’ documentation of sessions explained effects.
Method. Using an established model of prior students’ behaviors that predicted their final course grades, we applied that model to determine whether students had a greater likelihood of earning a B or Better or a C or Worse in the course. This dictated their eligibility for a digital training plus coaching intervention. The sample included 305 undergraduate biology students. Of participating students, 52.8% (N = 161) were flagged as more likely to earn a C or worse in the course; of these, 55.3% (N = 89) were randomly assigned to the training intervention and completed academic coaching sessions (Flagged Treatment + Coached) and 44.7% (N = 72) were randomly assigned to a control group and completed only the control activity (Flagged Control). The 144 students predicted to earn a B or better were assigned to a Non-Flagged group who completed the control activity.
Results. We found that training combined with coaching students who were flagged as likely to perform poorly (M = 72.67, SD = 15.81) significantly improved their final course achievement, compared to students who were also flagged but did not receive the training (M = 59.98, SD = 22.36). The Non-Flagged (M = 74.28, SD = 18.70) students also outperformed the Flagged Control students. In contrast, the Non-Flagged and Flagged Treatment + Coached groups did not differ statistically significantly on final course performance (see Figure 1). Correlations between language in coaches’ notes indicate key topics that may explain the effects obtained (See Table 1; models forthcoming).
Significance. When students were given the option to receive recommendations based on their own learning behaviors in the LMS in the early weeks of the course, and when these students responded to recommendations and sought academic success coaching, they earned similar grades to students who were predicted to perform well in the course. This timely, data-driven referral to digital training and online coaching led to a full letter grade difference between those who received and responded to training and referral, compared to a held out comparison group.

Authors

• Dan J. Gianoutsos, University of Nevada - Las Vegas

• MeganClaire Cogliano, University of Nevada - Las Vegas

• Matthew L. Bernacki, University of North Carolina Chapel Hill

• Jonathan C. Hilpert, University of Nevada Las Vegas

• Christy Strong, University of Nevada - Las Vegas