Evaluating a Digital App Vs a Facilitator-led Approach in a Remedial Program in El Salvador

• In Event: Remedial Education: Promising Platforms & Technologies

Proposal

The COVID-19 pandemic started as a health crisis, but quickly also became the biggest educational global crisis in recent years. This global crisis will have intergenerational consequences and requires coordinated efforts that can help reverse the learning loss. In El Salvador, learning loss due to COVID-19 complements an already existing learning crisis, with many girls and boys without capacity of developing foundational skills even after several years of schooling.
Catch-up Clubs (CuCs) is a short remedial education program designed to address the immediate educational needs of children in upper primary grades who are at greater academic risk. It has been piloted across three continents, eleven countries and has already reached over 30,000 children. It is an intersectoral holistic approach that harness the power of volunteerism and integrates elements of community-based learning, child protection and child poverty programming. Cath-Up Clubs have demonstrated significant impacts on children’s reading skills in experimental evaluations, with effect sizes between 0.37 and 0.71 standard deviations compared to control groups in only twelve weeks.

Building on the success of the CuCs model on improving reading levels, Save the Children is piloting a numeracy component in two districts in El Salvador. As a crucial foundational skill, adding numeracy to the curricula would broaden the scope and provide further tools for vulnerable children. However, a challenge with catch-up clubs model has been its operational complexity and burden- it has proven difficult in some contexts for volunteers to facilitate catch-up clubs with the regularly and intensity which the model proposes, and children and families themselves sometimes struggle to find the time to attend ninety sessions after school. Adding an additional full numeracy session could further exacerbate these challenges. For this reason, we proposed the use of a digital adaptive learning app, Onecourse, as a replacement to facilitator-led sessions to cover the new numeracy content. The tablet sessions will still be in-person and aided by a facilitator, but all the activities will be done through the App. Moreover, the session is 30 minutes long, compared to the 90 minutes facilitator-led session, and will be tagged on to literacy sessions.
The Onecourse app has already demonstrated a significant impact on math outcomes in low resource contexts, and yet little is known about how the impact compares to facilitator led activities. To test whether the short App-session produced results as good or better than the facilitator-led full session, we carried out a three-armed randomized controlled trial and a cost-effectiveness evaluation to measure impact in terms of learning outcomes and efficiency in terms of use of resources. This paper will present the results of the evaluation and provide insight into the effectiveness of using an adaptive learning app in comparison to facilitator led remedial math sessions. The findings and lessons learnt have significant potential to contribute to the knowledge base around remedial education and help inform policy-makers and practitioners in designing remedial programs.

Author

• Tim Murray, Save the Children