Evaluation at scale using a difference-in-difference approach

• In Event: Beyond RCTs: Developing and implementing right fit rigorous evaluation approaches toward improving children's foundational learning at scale

Proposal

Governments routinely scale-up interventions without rigorous evaluation. To understand causal impacts at scale, this often necessitates the use of non-experimental evaluation methods. Lalonde (1986) famously showed that non-experimental methods fail to generate reliable estimates when compared with experimental estimates. Yet, non-experimental methods have become substantially more credible in the past several decades. We leverage a unique dataset in India to compare these more credible non-experimental methods, such as quasi-experimental difference-in-difference techniques - to experimental randomized trial estimates in multiple studies in India. We find a low-cost application of differences-in-differences estimation yields highly credible results when compared with randomized trial estimates. We then use this method to conduct one of the largest scale evaluations of an education intervention to date in over 100,000 government schools in Uttar Pradesh, India. The intervention studied - teaching at the right level - generates large learning gains at scale.

Author

• Noam Angrist, Youth Impact; What Works Hub for Global Education, University of Oxford