Designing Staggered Difference-in-Differences Studies: Estimator Choice, Covariate Adjustment, and Power Analysis

• In Event: Advanced Methods for Causal Inference and Power Analysis

Abstract

Staggered difference-in-differences (DID) designs are increasingly used in education and policy research, where interventions often roll out to different units at different times. Yet design and analysis decisions in these studies are not straightforward. Recent work has shown that traditional two-way fixed effects can be biased under staggered adoption with heterogeneous treatment effects, and newer estimators differ in their assumptions, software implementations, estimands, and statistical performance. Practical guidance on power analysis remains limited, especially when treatment assignment is related to observed covariates and identification relies on conditional parallel trends.

This study brings together two connected lines of work on longitudinal quasi-experimental design using DID. First, we systematically review widely used staggered DID estimators and their implementation options in R and Stata, focusing on the choices that applied researchers must make, including control-group definitions, covariate adjustment, pre-treatment windows, and aggregation of cohort-time effects. Second, we extend this design guidance to settings with time-invariant covariates under conditional parallel trends, where standard fixed-effects specifications absorb covariate main effects and require estimator-specific modeling strategies. Across both settings, we develop power-analysis guidance using simulation-based methods and, for selected designs, closed-form variance expressions.

The study compares the major estimator families in current use: Callaway-Sant’Anna, Wooldridge/ETWFE, Borusyak-Jaravel-Spiess, Sun-Abraham, Gardner, Stacked DID, and de Chaisemartin-D’Haultfoeuille. We first simulate a multi-cohort staggered adoption design and then evaluate each estimator on bias, precision, type I error, power, and minimum detectable effect sizes. We consider both unconditional designs and conditional designs. The latter covers cases where treatment is associated with time-invariant covariates that also predict baseline outcomes and untreated trends. A worked example using a Texas district achievement panel shows that estimator performance varies meaningfully across estimands and design conditions. In the staggered DID comparison, BJS and ETWFE tend to have the strongest power. In the conditional-parallel-trends extension, power and type I error depend on overlap, baseline prediction, untreated trend associations, and estimand choice.

The paper contributes a unified framework for designing staggered DID studies in education and policy research. Rather than treating estimator choice, covariate adjustment, and sample size planning as separate decisions, we show how these decisions shape one another. The goal is to provide researchers with practical guidance on choosing among modern DID estimators, selecting substantively meaningful estimands, and planning adequately powered studies when adoption is staggered and covariates shape both treatment assignment and outcomes.

Author

• Wei Li, University of Florida
  Presenting Author