Paper Summary

Direct link:

Difference-in-Differences, Propensity Score Matching, and Propensity Score Weighting in School-Level Natural Experiments

Sun, April 11, 2:30 to 3:30pm EDT (2:30 to 3:30pm EDT), Division D, Division D - Section 2 Roundtable Sessions


Quasi-experimental methods are used to evaluate natural experiments. When aggregate, school-level pre- and post-treatment outcomes are available, methods of estimating the treatment effect include difference-in-differences (DiD), propensity score matching (PSM) or weighting (PSW), and propensity score augmented DiD methods. These are reviewed in an educational context and evaluated by simulation. Conditions include complete or partial knowledge of covariates that induce baseline differences and cause change over time in the outcome (violating the assumption of common trends). Confidence interval coverage rates are evaluated using cluster robust variance estimation (CRVE). A combination of PSW and DiD with CRVE is recommended across most conditions. The analyst is encouraged to consider not only baseline differences but how outcome related covariates may change over time.