Bringing monitoring and evaluation together: A case study from Sub-Saharan Africa

• In Event: Highlighted Session: Beyond Effect Size: Leveraging RCTs for Policy and Programming

Proposal

Monitoring and evaluation (M&E) of development education projects require substantial amounts of resources. As a reference, the United States Agency for International Development (USAID) devotes approximately 3 percent of total program dollars to external performance and impact evaluations (Sedrakian, 2016). This does not include resources for monitoring, which vary on a case by case basis, but which normally constitute between 3% and 10% of the program budget (Sedrakian, 2016). Considering that during the past four years, USAID has budgeted, in average, 772 billions per year for basic education (USAID, 2020), monitoring and evaluation of basic education alone costs between 46 billion and 100 millions per year. While most of the M&E resources are geared to feed indicators, output and outcome metrics, implementers and evaluators alike have an interest in getting more out of these investments and data.
In development education, one of the most important evaluation goals is to determine whether a project has had an impact on students’ outcomes. In this context, an impact evaluation is typically geared towards answering questions of the type – “was there an impact on students’ learning outcomes?”, “was this impact different across sub-groups?”, or “what other factors moderated the impact?”, to name a few. These evaluations can reach high levels of sophistication, with critical resources devoted to identifying causal impact, sampling and measuring student learning outcomes in appropriate ways. No matter how sophisticated, evaluation data alone rarely provides the ground through which to make meaningful connections back to the project. In the absence of monitoring data, the relationship between programs and impact can become a black box. While monitoring and evaluation data should be complementary, there is a tendency to split off evaluation from monitoring (Markiewicz & Patrick, 2016), preventing practitioners to use both sources of data to adequately meet multiple needs.
In this paper we support the notion that M&E activities should be closely aligned, and we show that doing so improves both monitoring and evaluation work. Specifically, we examine the properties of monitoring data that are optimal for contextualizing evaluation data, and the properties of evaluation data that are optimal for using along monitoring data. We do so in terms of methodological frameworks -qualitative, mixed methods or quantitative- and in terms of timing and content. We discuss ways in which to incorporate monitoring data in evaluation activities that enable stakeholders to participate and make use of evaluation results. Moreover, and drawing from other disciplines, we discuss innovative ways in which to use monitoring and evaluation data together and in a cost-efficient manner. We will illustrate these points using monitoring and evaluation data from a five-year intervention in a Sub-Saharan African country. Using different modeling techniques, we will show how project implementation mediates impact and helps identify clusters of schools that need different types of support. Implications for M&E work will follow.

Author

• Fernanda Gandara, School-to-School International