Poster #10 - Skill-Specific Preschool Curriculum Interventions: Examining Treatment Effect Heterogeneity with Distributional Methods

• In Event: 2-001 - Poster Session 1 during Breakfast

Abstract

Much of developmental and educational research has focused on examining treatment effect heterogeneity by estimating how a particular program or intervention influences different subgroups of children. The current study takes a different approach and uses distributional methods to identify variation in effects that have been previously hidden from a large multisite experimental study. The current study provides a new way for thinking about the effects of curricula on the distribution of children's outcomes by reevaluating the same experimental data from the original Preschool Curriculum Evaluation Research (PCER) study to test whether the original null mean effects found are concealing important distributional differences. Specifically, I investigate the possibility that the weak average effects of the PCER study disguise larger program effects for high- or low-achieving students. Prior research has only examined the average impacts of experimental curricula - the current study will use distributional methods to identify variation in effects that have been previously hidden. When designing curricular interventions, we should consider not just the average impact of curricula, but also the way it impacts the entire distribution of children's achievement. By estimating quantile treatment effects, this study explicitly tests five different possibilities of heterogeneous effects in early childhood curriculum interventions based on educational, developmental, and economic theory. Study findings suggest that skill-specific curricula have differential effects at various points across the distribution of children's literacy and language outcomes. I found the differential effects of skill-specific literacy curricula to depend on the outcome of interest. Specifically, I observed a compensatory effect in that children at the bottom of the distribution of Woodcock-Johnson Letter-Word benefited most from the treatment. I also found a skills beget skills effect for children at the top of the distribution on Woodcock-Johnson Spelling. For the domain of receptive vocabulary, as measured by the Peabody Picture Vocabulary Test, I found that results were consistent with the no effects hypothesis. The results from this study might have well been missed had it instead focused on estimation of average effects or subgroup effects. One of the goals of this study was to familiarize researchers with distributional analytic techniques, but it will still be important and policy-relevant for researchers to examine mean effects and heterogeneous effects on observable subgroups of children. This study is one example of how researchers can assess whether early childhood interventions are having their desired effects of boosting children's school readiness skills while narrowing achievement gaps.

Author

• Tutrang Nguyen, University of California, Irvine
  Presenting Author