Learning Recovery: An Impact Evaluation of a Math Phone-Tutoring Program for Lower Secondary School Students in the Peruvian Highlands

• In Event: Explorations of Educational Reforms and Tools for School Improvement

Proposal

In March 2020, Peruvian schools closed abruptly due to the COVID-19 pandemic. While schools have since reopened, Peru experienced one of the longest school closures globally (Torres et al, 2024), significantly worsening learning loss across the country, particularly among vulnerable populations. Access to education remains a major challenge for disadvantaged children. Even before the pandemic, only 17% of eighth graders in 2019 demonstrated proficiency in math (MINEDU, 2020). This figure dropped to just 11% in 2023, reflecting a six-percentage-point decline (MINEDU, 2024). The prolonged school closures, compounded by Peru's longstanding underinvestment in education for vulnerable regions, have deepened educational inequalities.

The Desafía-T Program
In 2023, to mitigate the learning losses in math, the Group for the Analysis of Development (GRADE), in collaboration with the Inter-American Development Bank (IADB), launched a pilot project called Desafía-T in the Ayacucho region of the Peruvian Highlands. The program provided remote tutoring via telephone for lower-secondary school students, with the goal of enhancing their math skills.

The intervention consisted of eight 30–40 minutes phone sessions, delivered by trained tutors. Students were divided into two groups:

 Group A: Students (both male and female) received math-focused tutoring.
 Group B: Female students received a combination of math tutoring and socio-emotional sessions designed to promote a growth mindset.

The program emphasized foundational math concepts such as place value, basic arithmetic, and problem-solving while encouraging students to reflect on their learning processes. The socio-emotional component, introduced to female students, aimed to boost self-confidence and motivation. The theory of change behind the program posited that combining math tutoring with growth mindset training would improve both academic and socio-emotional outcomes, ultimately fostering greater resilience and perseverance in learning.

Study Objective
The primary goal of this study was to evaluate the impact of the Desafía-T program on students' math skills and socio-emotional development, particularly growth mindset, in the Ayacucho region. The program aimed to help first- and second-year secondary school students recover math competencies that were not fully consolidated in primary school. The study sought to achieve two key objectives:

 Assess the impact on students' math skills.
 Evaluate changes in students' growth mindset, with a focus on female students.

Evaluation Design
The study employed an experimental design with random assignment of schools into treatment and control groups (30 schools per group). Data was collected at baseline and endline through surveys of students, teachers, and school principals, as well as math assessments for students. To account for variations in program participation, the evaluation estimated both the intention-to-treat (ITT) and the local average treatment effect (LATE). This allowed for a more nuanced understanding of the program’s potential and its impact on students who fully participated.

Results
The impact evaluation revealed significant improvements in both math skills and socio-emotional development for students who participated in the program.

In math, the intention-to-treat effect showed a positive and statistically significant impact of 0.08 standard deviations (SD). For students who attended seven or eight sessions, this effect increased to 0.20 SD.

Regarding socio-emotional development, the program had a notable positive effect on growth mindset, with an ITT effect of 0.17 SD. The effect grew with greater participation, reaching 0.39 SD for students who attended seven to eight sessions.

Policy Recommendations

Based on the program's outcomes, several policy recommendations were proposed:

 Cost-effective learning recovery: The success of remote, phone-based tutoring demonstrates its potential for addressing learning gaps, particularly in areas with limited internet access.
 Growth mindset interventions: The addition of growth mindset sessions for female students proved particularly effective in enhancing confidence and persistence. This highlights the importance of incorporating socio-emotional learning into academic programs.
 Tutor training and support: Tutors played a crucial role in the program's success. Therefore, ongoing training and support for teachers and tutors, especially in virtual or remote settings, should be prioritized.

References

MINEDU (2020). Resultados de las Evaluaciones Nacionales de Aprendizaje. Oficina de Medición de la Calidad de los Aprendizajes – UMC. Lima, Perú.
MINEDU (2024). Resultados de las Evaluaciones Nacionales de Aprendizaje. Oficina de Medición de la Calidad de los Aprendizajes – UMC. Lima, Perú.
Torres, I., Kloft, S., Kumar, M., Santosh, A., Pinto-Alvarez, M., & López-Cevallos, D. F. (2024). COVID-19 school closures in Latin America: untangling approaches impacting student health and wellbeing. Health Education.

Authors

• Juan Leon Jara Almonte

• Pablo Zoido