Improving education via evidence generation to inform implementation at scale: lessons from South Africa, Nepal and Botswana

Group Submission Type: Formal Panel Session

Proposal

Implementation research is scientific inquiry into questions of implementation to discover how and why implementation is going right or wrong, and testing approaches to improve/scale it. In practice, it may be called adaptive learning or entail less formal methods to respond to the inevitable challenges of implementation. In short, it is research that influences implementation. This resonates with the CIES 2023 theme of improving education for a more equitable world because implementation research can uncover nuance and formulate and test options for such improvement – at school, subnational and national levels. Any carefully planned implementation strategy can be improved via harvesting insights from actual implementation (Bauer et al, 2015).

We have many impact evaluations showing what worked, where, once; oft-cited findings get replicated at greater scale to work again but perhaps with less impact but the evidence base accumulates to be translated and tested elsewhere (Global Education Evidence Advisory Panel, 2020). In comparative and international education, we need more stakeholder perceptions, staff/partner observations, monitoring data and other sources to inform an intervention’s acceptability, appropriateness, and feasibility, as well as its adoption, fidelity, cost, coverage, and sustainability (Peters et al, 2013). Such data collection, analysis and reflection on implementation process and progress enables the critical balance of fidelity to core elements of a model with the fit and adaptation to local context (MEASURE, 2012). Where context includes “social, cultural, economic, political, legal, and physical environment, as well as the institutional setting, comprising various stakeholders and their interactions… the roles played by governments, non-governmental organisations, other private providers, and citizens” (Peters et al, p. 1), a focus on how these factors affect whether/how best pedagogical practice gets taken up and adapted is central to scaling and improving foundational literacy and numeracy (FLN).

We’ll focus in this panel on the use of implementation research for governments/programs scaling evidence-based pedagogies to achieve FLN. This resonates with the health sector’s wealth of experience in using implementation research to “improve the uptake, implementation, and translation of research findings into routine and common practices” (Padian et al, 2011). The beginning point is a proven pedagogical model for FLN achievement and the fidelity and fit are to new geographies and populations. Implementation research thus seeks to answer questions emerging from this process to offer the evidence base on which to improve fit and ensure uptake. This work centers implementation research as fuel for adaptive management - the intentional process of responding to continuous learning about project performance and environmental enabling conditions to achieve reform/intervention goals (Byom et al, 2020).

In the last dozen years or so, research has focused on FLN intervention impact while less attention or funding has looked at how to implement evidence-based pedagogical solutions at scale; even less has gone to how to respond to real-time implementation challenges with evidence-based models with feedback loops for contextualization/ adaptation. Through cases from South Africa, Nepal and Bostwana, this panel explores the size, shape and set up of current/recent experiences to learn from models and think about principles of good practice. We’ll learn from a model of nested RCTs in South Africa run from within the government in partnership with academics and donors. Then we’ll explore issues as they arose during implementation in Nepal by government and an international organization in a context of government-to-government and project bilateral funding. Finally, we’ll hear about how implementation drives rounds of A/B testing by a large national NGO in Botswana to optimize government scaling efforts. The discussion will center around whether the variation in this set of examples offers any principles of good practice and next steps for donors, implementers and researchers in supporting implementation research to optimize learning at scale.

Sub Unit

• Global Literacy SIG

Chair

• Benjamin Piper, Bill & Melinda Gates Foundation

Individual Presentations

• Lessons from the South African Early Grade Reading Study Series - Nompumelelo L Mohohlwane, Department of Basic Education

• The Teacher Professional Support System in Nepal: A case of implementation research informing policy - Wendi Ralaingita, RTI International

• Go big or go home: implementation driven A/B testing to drive learning at scale in Botswana - Tendekai Mukoyi, Youth Impact; Thato Letsomo, Youth Impact; Noam Angrist, Youth Impact