Trajectories of Fadeout in Cognitive and Social-Emotional Outcomes: A Meta-Analysis of Randomized Control Trials with Follow-up Assessments

• In Event: Projecting Intervention Impacts in Developmental and Educational Studies

Abstract

Child-directed interventions have long been designed with the intention of optimizing human development and potential. Contrary to theory that boosting children’s skills should lead to more advanced development, amassing evidence has suggested that treatment impacts commonly fade to null overtime (Bailey et al., 2017). Interestingly, some studies have found less evidence for fadeout in behavioral impacts (e.g., Dodge et al., 2015; Heckman et al., 2012). Indeed, boosts to social-emotional skills and beliefs may be more persistent than boosts to cognitive skills if they initiate larger positive feedback loops (Walton & Yeager, 2020) or if development of such skills is unlikely to otherwise happen in home or school contexts (see Bailey et al., 2017). It remains empirically under-investigated, however, whether impacts on social-emotional skills generally show less fadeout than impacts on cognitive skills.

Thus, the current meta-analysis investigated longitudinal trajectories of RCT treatment impacts on social-emotional versus cognitive outcomes. 426 papers were drawn from eight published meta-analyses on cognitive and social-emotional interventions. Of these, 203 studies were RCTs with cognitive of social-emotional outcomes, and 100 included follow-ups on the same students at least 6 months after post-test. 89 provided usable treatment impacts and were consensus coded for a robust array of study details, including 1,173 post-test and 2,164 follow-up effect size estimates. For the current analysis, the sample was limited to studies that reported impact estimates for the same construct (e.g., IQ) assessed using the same measure (e.g., Stanford Binet) at post-test and at least one follow-up. Preliminary data cleaning resulted in 472 cases where the same construct was measured using the same measure within the same study. 283 were cognitive outcomes (i.e., language and literacy, math, achievement, and cognition) and 189 were social-emotional outcomes (i.e., social-emotional, externalizing, internalizing, criminality, and substance use).

Figures 1 and 2 display impact estimate trajectories. Cognitive and social-emotional effects showed considerable, and similar, patterns of fadeout. On average, treatment effects on cognitive outcomes were .19 standard deviations (p<.001) at post-test, but dropped to .07 (p<.001) at 6- to 12-month follow-ups, and to nearly zero, and statistically non-significant, at assessments a year or more after post-test (ES = .02- .04). Social-emotional outcomes demonstrated similar trajectories, though initial post-test impacts were smaller than cognitive impacts (post-test= .06 (p<.001), 6-12 months= .04 (p<.001), 1 year and greater= .02-.04).

These results suggest that while RCTs tend to find substantive effects on cognitive and social-emotional skills at post-test, and smaller, but still statistically significant, effects soon after treatment end, these effects demonstrate fadeout overtime. Fadeout appears to be just as pervasive for social-emotional outcomes as it is for cognitive outcomes. Analyses are currently underway to further model these patterns. Indeed, these averages likely mask substantial heterogeneity which we will explore within the context of theories on fadeout and persistence.

Authors

• Emma Rose Hart, Teachers College, Columbia University
  Presenting Author

• Drew Bailey, University of California - Irvine

• Tyler Watts, Teachers College, Columbia University