Effects of an Early Mathematics Intervention on Stable and Time-Varying Components of Mathematics Achievement

• In Event: Academics in the Earliest Years of Formal Schooling: Building Evidence for Policy and Practice

Abstract

Purpose
The current study tested whether long-run treatment impact fadeout from a preschool mathematics intervention could be explained by a latent factor model that parsed the variation in long-run mathematics achievement measures into stable and time-varying components.
Perspective
Well-controlled correlational studies show a strong relation between children’s early mathematics skills and their later achievement (e.g. Duncan et al., 2007). Such correlational findings imply that if interventions can boost early mathematics achievement, the effects of such efforts may last many years. Unfortunately, recent evidence suggests that this may not be the case, as the treatment impacts of a successful preschool mathematics curriculum faded substantially in the years following the end of treatment (Clements et al., 2013). The current study hypothesized that such fadeout patterns could arise because the treatment affected time-varying aspects of mathematics achievement, but failed to affect factors related to mathematics achievement that were stable over time.
Methods
We relied on data from the TRIAD evaluation study (Clements et al., 2013), which evaluated the scale-up of the Building Blocks (BB) preschool curriculum. The study randomly assigned schools to one of two conditions: BB preschool curriculum or control (business as usual). Student-level mathematics achievement was assessed at the beginning (pre-treatment) and end (post-treatment) of preschool, and follow-up assessments were collected in kindergarten, first, and fourth grade.
We modeled the measures of long-run mathematics achievement from preschool through fourth grade as a state-trait model (see Bailey et al., 2014), in which we regressed a latent trait on the four post-treatment and follow-up measures of achievement. The trait loadings for this latent factor represented trait effects, or the amount of variance in the four measures that was stable over time. Within the model, we also regressed each measure of mathematics achievement on the previous measure; we refer to the resulting paths as “state effects” (amount of variation in achievement explained by changes in the previous measure). We then tested the effect of the treatment on both state and trait mathematics achievement.
Results
We found that the latent trait factor explained much more variation in mathematics achievement than the state effects, as trait loadings ranged from .76 to .94, whereas state effects ranged from .04 to .25. Further, we found that the treatment had no detectable effect on trait mathematics, but had a large impact on state mathematics (β= .53, p < .001), indicating that the treatment effects probably faded due to a failure to impact the stable characteristics that influence mathematics achievement over time.
When we tested our model in both the treatment and control groups separately, we observed larger state effects for students in the treatment group. This indicates that more transfer of knowledge occurred in the treatment group, but these differences were only present in the two earliest follow-up measures.
Conclusion
Our results suggest that a one-time intervention is probably not sufficient to affect mathematics achievement in the long-run, as such efforts are unlikely to affect the underlying factors that strongly influence achievement patterns over time.

Authors

• Tyler Watts, University of California - Irvine

• Douglas H. Clements, University of Denver

• Julie Sarama, University of Denver

• Christopher B. Wolfe, University at Buffalo - SUNY

• Mary Elaine Spitler, State University of New York

• Drew Bailey, University of California - Irvine