Persistence of Effects to Fifth Grade

• In Event: 58.026 - Longitudinal Evaluation of a Scale-Up Model: Critical Components and Effects on Persistence and Sustainability

Abstract

Objectives

The original study found that at the end of the preschool, kindergarten, and first grade years, both experimental groups, TRIAD-FT and TRIAD-NFT, significantly outperformed the control group at each time point. The pattern of impacts led us to examine whether impacts would persist into later elementary school years, several years after both arms of the intervention have ended (specifically in third, fourth, and fifth grades), and whether subgroups based on race/ethnicity and socio-economic status had experienced differential impacts.

Theoretical Framework

We posit that effects of building a foundation of comprehensive mathematics proficiencies and topics may be revealed when the demands of mathematics instruction increase. This alternative hypothesis, the latent foundation hypothesis, suggests that more demanding educational contexts (e.g., first and fifth grade) may be more likely to evince the long-term effects of early development of such comprehensive proficiencies.

Methods

At the start of the study (pre-K baseline), the study sample included 1,375 students. By the end of fifth grade, 781 students from the original sample were tested. While the original treatment and control groups were formed via random assignment and the attrition rates of the three groups were not substantially different (around 40 percent for each group), we formed the analytic samples in the 3rd, 4th and 5th grade analyses using propensity score matching to ensure the samples at the later grades were equivalent. Matching was conducted using baseline characteristics (pre-K pretest race/ethnicity, socio-economic status, gender, and disability) to yield tightly balanced groups on all characteristics (differences less than 0.1 standard deviations (SDs)). We estimated the long-term effects of TRIAD-NFT and TRIAD-FT using a 2-level logistic regression model that can be considered an elaboration of the basic Rasch item response model.

Data sources

Students’ mathematical knowledge was measured using the Tools for Early Assessment in Mathematics 3-5 (TEAM 3-5).

Results

In third and fourth grade, effects were not statistically distinguishable from zero for either group. In fifth grade we observed a striking “come-back”: effects were larger and statistically significant for both groups – effect sizes were about 0.26 SDs for FT and about 0.21 SDs for NFT.

For African-American students in FT, the estimated effects were statistically significant at all later grades (0.3-0.4 SDs). For African-Americans in NFT, effect estimates were insignificant in each grade. For White students in FT, the estimated effects were only statistically significant in fifth grade (0.57 SDs). For White students in NFT, the estimated impacts were significant in third and fifth grades (0.55 SDs in both grades). We see statistically significant effects for higher SES students in NFT in fourth and fifth grades (about 0.4-0.5 SDs) while for higher SES students in FT, only the fifth grade impacts were statistically significant (0.51 SDs). For lower SES students, neither condition had significant impacts in any of the later grades.

Scientific Significance

The re-emergence of significant effects at a critical point in elementary education—the transition to the increasing demands of the mathematics curriculum of fifth grade—supports the latent foundation hypothesis.

Authors

• Carolyn J. Layzer, Abt Associates Inc.

• Fatih Unlu, Abt Associates Inc.

• Douglas H. Clements, University of Denver

• Julie Sarama, University of Denver

• Christopher B. Wolfe, University at Buffalo - SUNY

• Mary Elaine Spitler, State University of New York