Investigating Heterogeneous Treatment Effects in Observational Studies Using Propensity Score Matching and Regression Tree

• In Event: Poster Session 2
  In Poster Session: Division D Section 2 Poster Session 1

Abstract

Identifying subgroups with heterogeneous treatment effects could help with incorporating students' diverse backgrounds into education decision/policy making. This paper proposes a new two-step approach to identify subpopulations with heterogeneous treatment effects in observational studies. In the first step, stratum-specific treatment effects are estimated by propensity score optimal full matching to obtain stratum-specific treatment effect estimates. In the second step, regression tree method is employed to further identify those subpopulations with significantly heterogeneous treatment effects differentiated by certain key covariates. A comprehensive case study with random permutation testing is conducted to evaluate the performance of the proposed approach. Results showed that the proposed approach could identify meaningful heterogeneous subgroups in observational studies with reasonable type I error rate.

Authors

• Jianshen Chen, Educational Testing Service

• Bryan Sean Keller, Teachers College, Columbia University