Endline stage – reflecting on the sustainability of program results after the end of implementation

• In Event: Measuring sustainability of school meals: research methods to measure sustainability at different stages of USDA Food for Education project implementation

Proposal

Save the Children´s IDEA (Investment for the Educational Development in the Highlands) project has been carried out since 2014 in six municipalities in the department of El Quiché, which is one of the most disadvantaged areas of the country. IDEA was designed to improve literacy levels in school-aged children in their mother tongue language and Spanish through training teachers in innovative approaches using the Literacy Boost methodology. It also aimed to improve health and dietary practices through donated commodities and funds from USDA McGovern Dole International Food for Education and Child Nutrition program and support from the Guatemalan Ministry of Education at the national, departmental, and local levels.

A mixed methods approach was employed for the project`s three evaluations (baseline, midline, and final) using the same research design and instruments. The final evaluation aimed to answer questions regarding the project’s relevance, effectiveness, efficiency, impact, and sustainability. Evaluation sustainability questions were “To what extent did the project plan and advance towards sustainability?” and “To what extent has project continuation been planned with activities, local appropriation, and development of sustainable partnerships?”

The Literacy Boost Reading Assessment (LBRA) was used to assess second and third grade reading abilities in Spanish, K’iche’ and Ixil; this included a student survey with questions about students’ socioeconomic situation, educational experience, home literacy environment, and knowledge and behaviors related to health, nutrition and hygiene. Focus groups were conducted with principals, teachers and parents to capture perceptions about students’ attention span, teaching methodologies, the level of organization for the food provision, organizations that support the schools, and ways to seek sustainability. Additionally, interviews were conducted with education authorities, mayors, and community leaders.

To understand if the literacy benefits of the program have been sustained from 2015 to 2018, a difference in difference model was used to compare student LBRA data from schools receiving the intervention in the first year of implementation with students receiving intervention in the second year. Additionally, regression models were employed to compare results from the baseline, midline, and final evaluation.

Results revealed an overall improvement from the baseline to the final evaluation for reading comprehension in second grade in all three languages and both geographical areas and an overall improvement in all K’iche’ and Ixil literacy components. Third grade literacy abilities also improved in both languages. Some literacy components showed a decrease in abilities from the midline to the final evaluation, pointing to the need for sustained efforts to support teachers to use innovative methodologies. These efforts have the potential to be sustained by the University of San Carlos, which has a diploma program to professionalize teachers in the Literacy Boost methodology in Spanish and local languages.

Results also revealed improved school attendance and students’ attention during class. Qualitative data pointed to the food provided, and type of food, being very influential contributing factors. Partnerships were developed with municipalities, gaining their support in paying the volunteer cooks. An effort must be made to continue this support to help ensure sustainability of the daily school meals.

Authors

• Frine Paz, Independent Consultant

• Kristin Rosekrans, Independent Consultant