Effects of a Professional Development Program on Third-Grade Reading Achievement in the Context of Colorado’s READ Act

• In Event: The Science of Teaching Reading Part 1: Educator Knowledge, Professional Development, and Student Reading Achievement

Abstract

Language Essentials for Teaching Reading and Spelling: 3rd Edition (LETRS) is a popular professional development (PD) program that is used by nearly 150,000 educators in the United States (Lexia, 2022), however, research on its efficacy is limited (Didion et al., 2020). We estimated the effect of LETRS on student reading ability by comparing average third grade reading achievement in schools that used LETRS to that of schools that used other evidence-based programs. To strengthen our causal claims and address pressing questions about the effects of PD, we also performed three a priori supplemental analyses (de Vocht et al., 2021; Didion et al., 2020). These analyses examined whether results differed for (a) early adopters of LETRS, (b) schools with initially low average levels of educator knowledge, and (c) late adopters of LETRS.

Based on the LETRS logic model and prior research (e.g., Desimone, 2009; Didion et al., 2020; Garet et al., 2008; Moats, 2009), we hypothesized that LETRS would improve educator knowledge of the science of reading, and in turn, classroom instruction, and average third grade reading achievement. We also hypothesized that LETRS would have a positive effect in schools with initially low average levels of educator knowledge, and no effect in schools that adopted LETRS late. We had no hypothesis about the effect of LETRS in early program adopters, but expected that their results may differ from those of the general population given that they often differ on important characteristics, like baseline performance (de Vocht et al., 2021).

Data for this study came from three sources. All LETRS data were provided by Lexia Learning LLC. Student assessment data and student demographic data were respectively obtained from the Colorado Department of Education (2019) and the National Center for Education Statistics' (2022) Common Core of Data. For all analyses, we used propensity score matching to establish baseline equivalence between the LETRS and comparison schools on both a pre-test and an array of recommended demographic variables (Pishgar et al., 2021; What Works Clearinghouse, 2022). We then regressed average third grade reading achievement on a pretest of the measure, school district, and propensity scores. As illustrated in Table 1, all analyses are expected to meet What Works Clearinghouse Standards (2022) with reservations.

Overall, students’ third grade reading achievement did not statistically differ for schools that adopted LETRS compared to other evidence-based PD programs. The sensitivity analyses similarly found that third grade reading achievement did not statistically differ for schools that (a) adopted LETRS early, (b) had educators with initially low average levels of knowledge, and (c) adopted LETRS late (Table 2). The results of this study are important because they suggest that LETRS, a widely used professional learning program, was no more effective at improving third grade reading achievement than alternative evidence-based programs in Colorado. Future research should investigate the extent to which specific professional development mechanisms and implementation context influence change in student reading outcomes.

Authors

• Brian M. Gearin, Lexia

• Rajendra Chattergoon, Cambium Learning Group, Inc.

• Young-Suk Grace Kim, University of California - Irvine

• Shayne B. Piasta, The Ohio State University

• Elizabeth Brooke, Lexia Learning