Poster #42 - Estimating the Treatment Effect Heterogeneity of Providing Performance Information in Education: A Causal Forests Approach

• In Event: Saturday Poster Luncheon

Abstract

Efforts to mitigate information asymmetry between parents and their children regarding academic performance have become a common practice among governments and K-12 educational institutions globally, with the aim of improving educational outcomes. These efforts are grounded in evidence from simple average treatment effect estimations, which suggest that, on average, providing parents with information about their children's academic progress can improve educational investments made by families, thereby enhancing children's academic achievement. However, the simple average treatment effect estimations are limited in their ability to identify critical heterogeneities in the changes of family educational investment that may result from such information interventions. That is, families exhibit significant variability in their characteristics, which may differently respond to such interventions and moderate their treatment effects on family educational investment. Understanding these heterogeneities is essential for large-scale policies like report card distribution. Some families may boost educational investment when informed, benefiting most from targeted interventions, while others may respond weakly or negatively, potentially limiting or reversing the impact.

This study employs a novel approach that integrates Randomized Controlled Trials (RCTs) with Machine Learning (ML) techniques to more accurately estimate the heterogeneous treatment effects of reducing information frictions between parents and children regarding academic performance, specifically on families' human capital investments. Methodologically, to eliminate selection bias that confounds the effect estimates, this study firstly conducts an RCT in 3,750 households with children enrolled in 4th to 11th grades in rural China, half of them (N=1,878) are assigned to the treatment group in which children’s academic information about their test scores in the midterm and final exams during the 2023-20224 academic year is provided. Households (N=1,872) in the control group are not provided with this information. The primary outcome variable of interest is family educational investment, including parents’ finance, time, and energy. Secondly, this study uses a nonparametric causal forests approach that was developed by Wager and Athey (2018) for heterogeneous treatment effect estimation of information-to-parents on family educational investment to achieve asymptotically normal and unbiased point estimates with valid confidence intervals. Therefore, the central aim of this study is to assess the treatment effect heterogeneity of information-to-parents on family educational investment by using a causal forests approach in the experimental setting.

This study makes three key contributions to the literature. First, it builds on two decades of research on academic information interventions and global report card policies by being the first to systematically examine the heterogeneous treatment effects of such interventions. The findings provide a scientifically rigorous and comprehensive understanding of the heterogeneous effects of these large-scale educational practices. Second, the study's precise estimation of heterogeneous treatment effects addresses a critical gap in the literature concerning the mechanisms of these interventions—specifically, how families respond to them, how they adjust their educational investments, and how these changes impact their children's academic outcomes. Third, this research contributes to the rapidly expanding body of literature that employs machine learning to enhance policy effectiveness and promote equity by accounting for heterogeneity and enabling targeted interventions.

Author

• Xinjie Zhang, Vanderbilt University
  Presenting Author