Individual Submission Summary
Share...

Direct link:

Understanding Mechanisms for a Sustained Program Impact: A Quasi-Experimental Evaluation of the McGovern-Dole International Food for Education Program in Mali

Mon, April 15, 10:00 to 11:30am, Hyatt Regency, Floor: Atrium (Level 2), Waterfront D

Proposal

Submitted By: Ms. Elnaz Safarha (Senior Research Analyst) and Dr. Kajal Gulati (Research Associate), IMPAQ International, LLC

Presentation Title: Understanding Mechanisms for a Sustained Program Impact: A Cohort-Comparison Quasi-Experimental Evaluation of the McGovern-Dole International Food for Education Program in Mali

Description of Presentation

Project Overview and Relevance to Sustainability
The United States Department of Agriculture has funded Catholic Relief Services (CRS) for three project phases of the McGovern-Dole (MGD) International Food for Education and Child Nutrition Program in Mali. The third phase, which is a five-year intervention, expands on the previously implemented MGD projects and aims to improve literacy of school-aged children in the Mopti and Koulikoro regions. Activities in the project include: providing school meals, take-home rations, and teacher/school administrator training on literary pedagogical techniques; establishing school management committees (SMC) and savings and internal lending community groups; and expanding illustrated report cards and distribution of deworming medications. In implementing these activities, CRS is building the capacity of the government at the national level to ensure a long-run sustained program impact.

CRS selected IMPAQ International, LLC (IMPAQ) to design and conduct performance and impact evaluations of the third phase of the MGD program (MGD III) at baseline (2016) and midline (2018). For MGD III evaluations, IMPAQ measured changes in outcomes and identified the program impact using quantitative and qualitative methods. The midline evaluation was completed in July 2018, and focused on students’ reading proficiency, attendance, parent/community engagement, as well as safe food preparation knowledge and storage practices. IMPAQ collected survey data from students, caregivers, teachers, principals, and school management committees, as well as qualitative data through interviews and focus group discussions.

In this presentation, we will describe the quasi-experimental cohort comparison design for understanding the program’s causal impact on children’s literacy outcomes. A long-run and lasting improvement in children’s literacy outcomes is one of the most critical measures of the program’s sustainability. The presentation will highlight the quasi-experimental methodology used to isolate the mechanisms underlying these sustained changes in children’s literacy outcomes. We will share our experience on setting-up rigorous evaluation designs for capturing these long-term sustainable changes in children’s educational attainment in Mali, which is of direct relevance to the conference theme, “Education for Sustainability.”

Methodology and Data Sources
To measure the causal impact of the program on improving children’s literacy outcomes, we employed a quasi-experimental cohort comparison method. The methodology measures the two- and three- year effects of having exposure to teachers trained in the balanced literacy approach (BLA) on improvements in children’s literacy as captured by students’ reading assessment scores. The design uses earlier cohorts as a comparison group to later cohorts, and controls for any time effects that could be influencing our results simultaneously. We capture the effect of two years of exposure to BLA-trained teachers on grade 2 students (treatment group) at midline as compared to grade 2 students (comparison group) at baseline. The grade 1 students from midline and baseline are used to capture the time effects of the program intervention between baseline and midline. Similarly, we estimate the three-year program effects by using grade 3 student data at midline and baseline. In order to implement the quasi-experimental design, we collected data from 2,506 students (1,216 boys and 1,290 girls) and 189 teachers.

Key Research Insights
Our cohort comparison design shows a clear proof of program impact: children’s literacy outcomes improved by one level due to two years of exposure to a BLA-trained teacher and increased by more than a level and a half due to three years of exposure to BLA-trained teachers. These findings are robust to adding other covariates, such as the student’s gender. However, we do not find any evidence of indirect spillover effects indicating the mechanism of success: children who receive direct, intensive, and prolonged exposure to well-trained teachers are able to improve their reading proficiencies.

Research Contributions
The research makes unique contribution to our understanding of what works in improving children’s educational attainment and sustaining its long-term effect, especially in extremely poor, politically unstable contexts. From a policy perspective, the research demonstrates the critical importance of investing in teachers by providing them training in teaching techniques and pedagogies. The findings suggest that even in areas with inadequate school infrastructure and services, children’s learning outcomes has the potential of improving when exposed to effective teachers. Such investments in teachers are, therefore, a cost-effective means of bringing about sustainable changes in children’s educational outcomes. From a methodological perspective, the research design is unique in that we are able to causally understand the mechanisms driving program impact by taking advantage of the schooling set-up prevalent in the study area and despite the challenges imposed by attrition of schools due to political turmoil.

Authors