2019 Dissertation Award: Causal Mediation Analysis in Multisite Trials

• In Event: AERA Division D Invited Session for 2019 and 2020 Awardees

Abstract

There has been an increasing use of multisite randomized trials in evaluations of educational programs. Multisite designs provide unique opportunities for investigating between-site heterogeneity in the mediation mechanism that characterizes an educational process central to a program theory. However, such opportunities have not been fully utilized due to some important constraints of existing analytic tools. To enable researchers to assess the generalizability of an education program theory across a wide range of contexts, Dr. Qin’s dissertation developed a new analytic procedure that enhances both the external validity and internal validity of multisite causal mediation analysis. This includes incorporating a sample weight to adjust for sample and survey designs, employing a non-response weight to account for non-random attrition, and utilizing a novel propensity score-based weighting strategy to flexibly decompose the average program impact at each site into indirect effects transmitted through hypothesized focal mediators and a direct effect attributable to all the other pathways. Extending a theoretical model of causal inference, Dr. Qin defined, identified, and estimated the population average and the between-site variance of the direct and indirect effects and developed a sensitivity analysis strategy to assess the consequences of potential violations of key identification assumptions.

Xu Qin is an Assistant Professor in Research Methodology at the Department of Health and Human Development of the School of Education at the University of Pittsburgh. She holds a Ph.D. from the Department of Comparative Human Development at the University of Chicago. Her research focuses on solving methodological problems in causal mediation analysis and evaluating educational programs and policies. In her dissertation, she developed methods for investigating causal mediation mechanisms in multisite evaluations with a focus on quantifying between-site heterogeneity in the mechanisms. She also employed the methods in an in-depth evaluation of Job Corps, the nation’s largest education and training program for 16-24-year-old disadvantaged youths.

Author

• Xu Qin, University of Pittsburgh