Unpacking Professional Development Impact: Dosage, Implementation, and Student Outcomes in Early Math

• In Event: Math for All: Math Games and Activities to Improve Learning and Attitudes Across Diverse Contexts

Abstract

This study investigates how variation in professional development (PD) dosage and school-level implementation quality mediate the effectiveness of a three-year PreK-2 mathematics intervention. Building on prior quasi-experimental findings demonstrating significant treatment effects (Authors, 2022, 2023), we examine two critical questions: first, how cumulative student exposure to PD-trained teachers influences learning outcomes, and second, how structural implementation components shape program success. Our work addresses a pressing gap in understanding the mechanisms through which sustained PD translates into measurable student achievement gains, particularly in under-resourced urban settings.
Grounded in Durlak and DuPre's (2008) implementation science framework, this study extends theoretical models of PD effectiveness by integrating Century and Cassata's (2016) implementation measurement approach with dosage principles. We conceptualize teacher development as a multiplicative function of both individual participation intensity and organizational support systems, while drawing on learning trajectory research (Clements et al., 2008) to assess content-specific changes in student outcomes. This framework allows us to bridge the gap between teacher quality studies (e.g., Sanders & Horn, 1998) and implementation science.
Our mixed-methods analysis draws from three years of intervention data across 15 urban schools. Using three-level hierarchical linear modeling (students nested in classrooms nested in schools), we developed two key metrics: an "effective dosage" measure combining PD hours with student exposure duration, and a weighted implementation composite score (ICC = 0.93) quantifying training exposure, leadership support, and instructional practices. These measures were applied to multiple outcome datasets, including REMA assessments (Dong et al., 2021), district benchmarks, and state standardized tests, while controlling for teacher mobility patterns documented in district administrative records.
The results reveal three significant patterns. First, we found robust dosage effects, with each additional year of student exposure to OGAP-trained teachers predicting measurable gains across all assessments (REMA β = 0.24, p < .01; STAR-Math β = 0.18, p < .05; state tests β = 0.12, p < .05). Second, treatment effects persisted even when students transitioned to non-OGAP teachers, suggesting durable impacts on both teacher practice and student learning. Third, while school-level implementation scores showed limited predictive power, this likely reflects our measurement focus on structural supports rather than classroom-level instructional quality, combined with high baseline implementation consistency across schools.
These findings make substantial contributions to both research and practice. Scientifically, we advance implementation science by demonstrating how dosage effects accumulate independently of organizational factors, while methodologically, we validate an approach for quantifying PD exposure in multi-year interventions. Practically, our results underscore the importance of investing in content-focused, sustained teacher development (Darling-Hammond et al., 2017; Lynch et al., 2019) while highlighting the need for district-wide scaling to maximize student access to improved instruction. The study particularly emphasizes moving beyond simple treatment-control comparisons to understand how intervention components interact with school contexts (Conaway et al., 2022), offering a model for evaluating complex educational interventions.

Authors

• Caroline Brayer Ebby, University of Pennsylvania

• Anahita Kumar, University of Pennsylvania

• Karina Diaz, University of Pennsylvania