Who Benefits From Additional Online Practice Opportunities in a Gateway Math Course?

• In Event: Cognitive, Metacognitive, and Motivational Interventions Supporting Student Success in STEM: What Works, When, for Whom?

Abstract

Background/Aims
Practice testing is one of the best-researched learning techniques in the educational sciences and has shown considerable promise in boosting student learning across diverse settings (Yang et al., 2021). However, surprisingly little research has evaluated the effectiveness of self-testing in demanding math courses in higher education (Carvalho et al., 2022; Wong et al., 2017), which are often plagued by high dropout rates and low performance (Benden & Lauermann, 2022). Moreover, self-testing in mathematics is conceptually different from fact recall in that it is about knowing how to solve a problem rather than memorizing the solution. The present study thus examined (a) the effectiveness of self-testing via two online practice opportunities in a gateway math course, and (b) the extent to which different subgroups of students benefitted from self-testing.

Method
Economics/business administration students (N=188, 59% female) enrolled in a gateway math course participated in the study. Students participated in three 30-minute online self-testing sessions, each covering content from the previous 3-to-4 weeks. A daily self-testing app was introduced in the second half of the course to practice content from the first half. Students received immediate corrective feedback. Students' gender, prior performance, motivational beliefs, personality traits, time preferences, and course goals were assessed at the beginning of the semester (Table 1).

Results
First, OLS regressions revealed that self-testing with the practice tests improved students' exam scores by about 5 points (of 90). A double-selection regression (Belloni et al., 2014) was then used to select relevant control variables, reduce a potential omitted variable bias, and avoid overfitting in the prediction of exam performance; the self-testing effect remained significant but decreased to 2.5 points (Table 2). Use of the daily self-testing app had a significant effect on students' exam performance in the simple OLS regression, but not in the post-double-selection regression.

Second, heterogeneous practice effects were examined for a set of LASSO-selected control variables (Table 2). A post-LASSO regression showed that risk-averse students, those who planned repeated practice, and students with a higher math self-concept were most likely to benefit from self-testing. Notably, students' gender, achievement goals, and personality traits did not contribute to differential practice effects. Only one significant interaction emerged for self-testing via the daily app; higher open-mindedness corresponded to greater benefits from self-testing.

Finally, a quantile regression revealed that self-testing via the practice tests was most effective for students who obtained a lower end-of-term exam grade (see Figure 1).

Discussion
Our results show that self-testing is a promising intervention to support students' academic success in demanding math courses. This intervention appears to support those who need it most—low-performing students. In contrast, students' gender, achievement goals, and personality traits were not associated with differential self-testing effects, which supports the broad applicability of self-testing. The modality of implementation, however, warrants further consideration, as course-embedded practice tests were more effective than a daily self-testing app.

New data from an ongoing follow-up study will also be presented, which uses within-person randomization to assess content- and time-specific effects of self-testing on later performance.

Authors

• Jakob Schwerter, TU Dortmund University
  Presenting Author

• Thomas Dimpfl, University of Hohenheim
  Non-Presenting Author

• Johannes Bleher
  Non-Presenting Author

• Kou Murayama, University of Tübingen
  Non-Presenting Author

• Fani Lauermann, University of Bonn
  Non-Presenting Author