Fidelity of implementation of early grade reading programs: What to measure? And is there a link with learning outcomes?

• In Event: The relationship between fidelity of implementation and learning: what the data tells us

Proposal

In recent years, early grade reading (EGR) programs have become a focus of significant investment in the international development community. To better evaluate these programs, we believe that it is critical to understand the extent to which they are implemented according to design, in particular at the classroom level, to enable identification of gaps in programming and take steps to improve implementation. To do this FHI 360 has developed comprehensive protocols for measuring fidelity of implementation (FOI), particularly as it applies to adherence by teachers to scripted or semi-scripted lesson plans. This involves conducting lesson observations using a tablet-based tool that is customized to the EGR curriculum and allows the observer to record the extent to which lesson activities are implemented the way they were designed. The FOI monitoring system also measures whether other elements of the program are being implemented as intended, such as coaching, the use of specific teaching and learning materials and the use of languages of instruction through observation and interviews. In most EGR projects, the assumption is that if teachers and schools implement the lessons and follow the program as intended, student reading outcomes will improve, following the theory of change. But is that the case?

In this presentation, we will present recent research examining different models of FOI monitoring, discussing stakeholders’ (teachers, head teachers, enumerators) perceptions of what should be monitored through an FOI system and providing analyses of FOI and learning outcomes data from three different EGR projects: the USAID/Ghana Learning Activity, the USAID/Madagascar Mahay Mamaky Teny (MMT) program and the UNICEF-funded Reading and Numeracy Activity (RANA) in Nigeria. The presentation will provide recommendations on the components of an effective FOI monitoring system for EGR programs from a sustainability and scalability standpoint and discuss whether positive relationships are found between FOI and reading outcomes. This presentation raises and aims to answer the following questions: how can we test the theory of change of EGR projects? What elements of the intervention matter and should actually be measured? How should we measure FOI and design a FOI monitoring system?

Authors

• Yvonne Cao, FHI 360

• Wael Moussa, FHI 360