Improving Early Literacy Through Game-Based Learning and Adaptive Personalization

• In Event: A Structured Approach for Testing Effectiveness of Educational Technology Products

Abstract

Research has shown that children reading below grade level at the end of 3rd grade are four times more likely to drop out of school (Hernandez, 2011). However, in the United States, one out of 10 children in classrooms are English learners (ELs), and they are one of the fastest growing segments of the K-12 student population (NCES, 2022). [Product] is an app that supports the vocabulary development of K-5 students, including English Learners. The program is guided by a working theory that supports students’ vocabulary development through engaging word puzzles, game-based learning, student-adaptive personalization, and tailored educational content. The intended outcomes of [product] are supported by empirical evidence of personalization, visual cues, and game-based learning on student outcomes. Guided by this program theory,[research organization] conducted several formative tests of the product, which resulted in iterative improvements to the product.
The present work describes a recent randomized control trial designed to explore the evidence of promise of the [product] app’s ability to affect students’, including ELs’, knowledge of vocabulary and spelling, their attitudes towards vocabulary learning, as well as evidence of the usability, feasibility, and fidelity of implementation of [product]. Teachers were randomly assigned to either a treatment of implementing [product] in their classes as part of their required English language instruction or to continuing their business-as-usual practices. There were a total of 264 third grade students across thirteen teacher’s classrooms from California and New York in the final analytic sample (Tx n = 133 and Cx n = 131) and 82 were classified as EL
All teachers received training at the start of the pilot study on how to complete study tasks and treatment teachers received additional training on how to integrate the [product] app into their normal classroom instruction. Treatment teachers were asked to incorporate the [product] app into their existing direct English language instruction at least three times a week for 10 weeks and support students with 30-minutes of individual [product] app usage. Teachers were asked to complete pre- and post-study surveys, participate in post-study interviews, and administer pre- and post-student assessments and surveys.
We ran a 2-level hierarchical model on our researcher developed proximal assessment, accounting for nesting of students within teachers (a random effect), and including fixed effects for condition and pre-test. The model estimates that going from the control to the treatment condition results in a 0.91 change in post-test score, which is statistically significant at p < .05 and corresponds to an effect size (Hedges’ g) of 0.34. These results provide evidence that [product] improves student vocabulary development, and validate the importance of early formative testing guided by the working program theory.

Authors

• Lia Francis-Bongue, WestEd

• Celia Gleason, WestEd

• Jessica Triant, WestEd

• Bryan Matlen, WestEd

• Elli Suzuki, Kooapps

• Chun-Kai Wang, Kooapps LLC