The relationship between fidelity of implementation and learning: what the data tells us

Group Submission Type: Formal Panel Session

Proposal

Most international education projects involve some form of trainings and capacity building, for teachers, education officials or even community members. The assumption is that these trainings and capacity building activities lead to increased knowledge, and improved practices and behaviors, which in turn lead to improved student outcomes, and in particular learning outcomes. However, few studies assess the intermediary outcome of “improved practices and behaviors” which makes it difficult to evaluate whether the training and capacity building activities have been effective. On the other hand, when student learning outcomes do improve, it is important to understand the extent to which this was due to improved practices and behaviors. In the context of early grade reading programs, this means understanding the relationship between learning outcomes and improved teacher or coaching practices such as teacher’s use of specific instructional tools or lessons plans, or the frequency of coaching received by teachers. Understanding the specific teacher or school practices that have a clear relationship with learning outcomes enables program implementers to adapt their intervention to focus on these critical elements. And understanding what trainings and capacity building activities lead to improved teacher or school practices also allows program implementers to modify their capacity building activities to better promote behavior change that is lasting and sustainable.

The three presentations in this panel explore measurement of uptake of targeted practices and behaviors using a fidelity of implementation (FOI) monitoring framework. In the first presentation, the presenter will discuss FOI at the systems-level and focus on uptake of capacity building activities particularly by education officials and administrators, as measured qualitatively through the Kirkpatrick’s Learning Evaluation Model. The second presentation will shift gears to focus on measuring uptake and FOI at the classroom and school-levels, and explore the relationship between FOI and learning outcomes. Finally the third presenter will also delve into the relationship between FOI and learning outcomes, and in addition will discuss recommendations for implementing comprehensive FOI monitoring systems. The presentations will touch on data use to create more sustainable impacts.

Sub Unit

• General Pool

Chair

• Carina Omoeva, FHI 360

Individual Presentations

• Examining fidelity of implementation at the systems-level: Assessing learning and uptake from capacity-building events among diverse stakeholders - Juan Luis Córdova, USAID/LAC Reads Capacity Program Regional Director Juárez and Associates

• 2nd Generation Multi-Country Project Implementation Monitoring System: What have we learned about implementation fidelity, program quality and reading outcomes? - Pinaki Joddar, Room to Read

• Fidelity of implementation of early grade reading programs: What to measure? And is there a link with learning outcomes? - Yvonne Cao, FHI 360; Wael Moussa, FHI 360

• Collective impact approaches to promoting early childhood reading development for disadvantaged children around the globe - Kouider Mokhtari, University of Texas at Tyler; Mariam Britel-Swift, US Agency for International Development (USAID); Mariam Dahbi, Harvard University; Mohammed Elmeski, American Research Institutes

Discussant

• Amy Mulcahy-Dunn, Encompass LLC