Accelerating Early Math Learning with a Digital Math Resource: A Cluster Randomized Controlled Trial

• In Event: 3-101 - Using Education Technology to Systematically Enhance Teaching and Learning Across Grades and Content Areas

Integrative Statement

Introduction
With only 40% of students proficient in math by fourth grade, the U.S. faces an acute need to scale innovative approaches to help close the math achievement gap (NCES, 2018). Yet, there remain few evidence-based and engaging digital learning resources to accelerate mathematics in early learning. Furthermore, while digital resources are becoming ubiquitous, continued research is needed to understand if and how they impact children’s learning outcomes. To address this need, developer researchers partnered with independent researchers to conduct a cluster randomized controlled study to evaluate the impact of a research-based, supplemental digital math learning resource on kindergarten students’ early number sense.

Hypothesis
[The digital math resource] is an adaptive game-based curriculum designed to help children build a strong understanding of fundamental number sense concepts, the numbers 1-20, and the quantities for which they stand. With engaging characters and scenarios, individualized learning pathways, and continuous assessment built into every level of every game, [math app] seeks to help children achieve proficiency through practice that incorporates repetition and variation. We predicted that students who use [math app] would see an increase in math performance over a control group of students receiving business-as-usual instruction.

Study Population
A total of 453 students (50.7% female, 76% Hispanic, 22% African American, M age=5.44 years) participated in the study. They came from 20 kindergarten (K) and transitional kindergarten (TK) classrooms at four Title I elementary schools in urban Southern California.

Methods
This study had a blocked cluster random assignment design, in which classrooms within each school were randomly assigned into the treatment group or the control group. Treatment classrooms received full access to the [math app] in fall 2017 and were asked to implement the app in small groups for 15 minutes/day, three days/week. Control classrooms conducted business-as-usual instruction. Selected TEMA items were administered in the child's preferred language. There were no statistically significant group differences in attrition, age, gender, or test language.

Results
To control for students’ performance at pretest, we used a 3-level hierarchical linear model (HLM) to account for differences by students based on their pretest score, group assignment, and school. Results confirmed that, overall, students who used the math app outperformed their control group peers on selected TEMA-3 items by 5.71% at posttest (effect size=.23; see Figure 1).

Notably, students did not have to devote large quantities of time to using the app to achieve these effects (M=5.22 hrs. total). Additional analyses revealed that the impacts of [math app] are greatest for those students who have some prior, basic number sense and/or who were more engaged in using the app. Importantly, [math app] had the greatest impacts on the most difficult, and most likely to be overlooked by teachers, math skills for young learners (Engel, Claessens & Finch, 2013; see Figure 2). Lastly, teachers reported having a positive experience with and recognized the value of [math app] as a resource to personalize learning and one that they want to continue using. Implications for policy, practice, and future research will be discussed.

Authors

• KP Thai, Age of Learning
  Non-Presenting Author

• Abby Schachner, Age of Learning, Inc.
  Presenting Author

• Linlin Li, WestEd
  Non-Presenting Author