Difference-in-Differences, Propensity Score Matching, and Propensity Score Weighting in School-Level Natural Experiments

• In Event: Matching and Selection in Observational Data

Abstract

Quasi-experimental methods are used to evaluate natural experiments. When aggregate, school-level pre- and post-treatment outcomes are available, methods of estimating the treatment effect include difference-in-differences (DiD), propensity score matching (PSM) or weighting (PSW), and propensity score augmented DiD methods. These are reviewed in an educational context and evaluated by simulation. Conditions include complete or partial knowledge of covariates that induce baseline differences and cause change over time in the outcome (violating the assumption of common trends). Confidence interval coverage rates are evaluated using cluster robust variance estimation (CRVE). A combination of PSW and DiD with CRVE is recommended across most conditions. The analyst is encouraged to consider not only baseline differences but how outcome related covariates may change over time.

Author

• Peter Boedeker, Baylor College of Medicine