Estimating Cross-Site Impact Variation to Improve the Statistical Power for Multisite Randomized Trials in Education (Poster 5)

• In Event: Innovations in Statistical Power, Intervention Planning and Design, and Causal Effects

Abstract

This study empirically estimated the cross-site variation in the impact of educational interventions in a set of completed large-scale MSCRTs funded by the Institute of Education Sciences (IES) to inform statistical power analyses for MSCRTs in education. The study used hierarchical linear models to estimate cross-site impact variation and PowerUpR package in the program R to demonstrate the application of the estimates to power analyses. The results indicated that the magnitude of cross-site impact variation of educational interventions ranged from 0.09𝜎 to 0.37𝜎. The study found that some interventions had consistent near-zero impacts across sites while others had substantial cross-site impact variation. The results can be leveraged by education researchers to design future MSCRTs with greater precision and efficiency.

Authors

• Thuy Dung Pham, University of Georgia

• Jessaca K. Spybrook, Western Michigan University