BackHome
Refresh: OffView Personal Schedule
Session Submission Summary

Implementation Measurement: How to capture and improve delivery of education programs

Sat, March 28, 4:30 to 5:45pm, Hilton, Floor: Ballroom Level - Tower 3, Continental 8

Group Submission Type: Workshop

Description of Session

Much of the variation in how effective education interventions are at improving learning can be explained by how well the program was implemented (Angrist & Meager, 2023). For studies with null or negative results, it is often unclear whether the failure was due to flaws in the intervention’s design or shortcomings during implementation. Despite this, implementation continues to be under-reported – only 12% of education development studies in education from 2019 to 2022 reported implementation metrics or Treatment-on-the-Treated (TOT) effects. Amid tighter international aid budgets, regular, systematic implementation measurement is more essential than ever. Making implementation more transparent – turning the ‘black box’ into a map of who does what and where it can break down – can inform better decisions about the design, implementation and scale-up of effective education programs.

However, given the wide variety of education interventions, it is not always clear what it means for a program to be implemented ‘well’. In this workshop, the What Works Hub for Global Education will outline a practical approach with useful tools and common principles to help researchers and implementers track and compare implementation across a range of education programs. Our simple designed/delivered/received framework facilitates comparisons between what was planned, what was actually delivered, and how both align with established best practices. Building on this, we will cover how to anchor relative measures of implementation quality in direct observations of what students ultimately receive. Participants will work with guidance on how to define and capture key dimensions of implementation fidelity, such as quantity and quality. The session will also cover potential measurement tools, principles for designing new ones, and key trade-offs to consider.

The workshop will combine a presentation, group-based learning activities and peer discussion. Guided by the What Works Hub for Global Education, facilitators representing a mix of system-level organizations (e.g. Gates Foundation, the World Bank) and implementers (e.g. Pratham, Youth Impact), will support attendees towards completing each activity. By the end of the workshop, attendees will leave with a structured framework to guide their thinking on implementation within their own contexts, along with practical tools and strategies to measure it effectively.

Sub Unit

Chair

Workshop Organizers

Presenters

Discussants