Poster #229 - Mathematics Learning using ALEKS in K-12 and Higher Education: A Meta-Analysis

• In Event: 3-206 - Poster Session 13
  In Poster Session: PS 13 Section - Technology, Media & Child Development

Integrative Statement

Increasingly, technology has begun to assume an integral role in education. The U.S. Department of Education (2017) argued that technology has the transformative power to eradicate enduring gaps in achievement and accessibility to quality education. Adaptive technologies, or technologies that personalize instruction based on the individual student, are one such powerful tool. The increased use of educational technologies will have a profound impact on the way in which current and future generations of children learn. Yet, where there is great promise, there is also concern about the potential negative effects of educational technologies. The current study examined the effects of Assessment and LEarning Knowledge Space (ALEKS), a commonly-used adaptive learning system. Empirical studies of the effects of ALEKS on mathematics achievement have produced mixed findings. As such, a meta-analysis was conducted to examine the effects of ALEKS on mathematics learning.

This study was part of a larger meta-analysis of the effects of adaptive learning technology in science, technology, engineering, and mathematics (STEM) education. Standard procedures were followed to search, identify, and code eligible research studies. High coder reliability was achieved on all coding categories. Final data analysis included 37 primary studies. Hedge’s g was used as the effect size measure. Dependent variables included: (1) cognitive outcomes, such as course grades, (2) affective outcomes such as mathematics anxiety, enjoyment of learning, motivation, and (3) behavioral outcomes such as effort, time on task, involvement, and attendance. Moderators included learning context (e.g., level of education, course type) and methodological characteristics (e.g., research design, measures). All analyses were conducted using the R package metafor.

Our results indicated that there were positive effects of ALEKS on affective (g = 0.20, SE = 0.14, p = .15, 95% CI [-0.07, 0.47]) and behavioral outcomes (g = 0.68, SE = 0.49, p = .17, 95% CI [-0.28, 1.63]). For cognitive outcomes, ALEKS users outperformed students who received traditional instruction (e.g., traditional lectures, paper-and-pencil based assessments, teacher-led classes, or direct instruction) by 0.18 standard deviations (g = 0.18, SE = 0.07, p = .01, 95% CI [0.04, 0.32]). Moderator analyses indicated that ALEKS effects were stronger in higher education (g = 0.30 vs. g = 0.10 in K-12 schools), in supplementary instructions (g = 0.43 vs. g = 0.14 in regular courses), and in pretest-posttest-control groups designs (g = 0.46 vs. g = 0.06 in other types of designs).

Given the popularity of ALEKS and adaptive learning technology in all levels of education, the current study provided a much needed meta-analysis of empirical studies. We found a small advantage of ALEKS over traditional instruction in enhancing student performance in mathematics courses, as well as potential benefits of ALEKS on positive affect and behavior. ALEKS is low cost, accessible, and easy to use, making it a viable option for schools to consider as an alternative to traditional instruction. It provides automatic and personalized feedback to students and thus can alleviate some of the burden on the instructor, especially for large, heterogeneous classes in higher education.

Authors

• Shuyan Sun, University of Maryland Baltimore County
  Presenting Author

• Allison French, University of Maryland, Baltimore County
  Non-Presenting Author

• Rebecca Dowling, UMBC
  Non-Presenting Author

• Linda Hodges, University of Maryland Baltimore County
  Non-Presenting Author

• Nicole Else-Quest, University of Maryland-Baltimore County
  Non-Presenting Author