Catch-up Clubs (CuCs) Sponsorship Evaluation in Bangladesh

• In Event: Pedagogies & possibilities: Evidence from literacy studies across contexts and spaces

Proposal

Presenters:
Roxana Khanom, Mohammad Sarwar Basher and Hosneara Khondker

Catch-up Club is a holistic approach that integrates elements from other programmes that have successfully demonstrated rapid learning gains and builds on Save the Children’s existing expertise on community-based learning, child protection and child poverty programming. Catch-up Clubs are designed to address learning loss due to school closures during the COVID-19 pandemic and prior challenges through providing catch-up learning in literacy to children aged 8-13 who have not yet acquired literacy at the level for the last grade of lower primary school (often grade 2). The activities provide the learning boost and social support that supports students most affected by inequality and discrimination to catch-up on learning and stay in school. Community-based levelled learning clubs built on the evidence-based Teaching at the Right Level (TaRL) approach and Save the Children’s Literacy and Numeracy boost common approaches.
The main purpose of the evaluation was to measure the impact of Catch-up Clubs on literacy outcomes and contribute to the understanding of how effective and scalable the CuCs model is to addressing learning loss. The evaluation was also meant to inform on project technical improvements and strategic internal decisions regarding funding and project scale-up; demonstrate track record in remedial support to external bodies; inform governments and donors on potential strategic scale-up: and contribute to learning in the remedial education sector. The scope of the project was limited to the districts of Gaibandha and Barishal in Bangladesh. Data was collected from girls and boys in grades 3 to 5 who were not yet at the grade 2 literacy level as per the CuC Assessment, along with their caregivers in target schools. Comparison groups of children and their caregivers who match the criteria but were not enrolled in the CuC were also engaged from schools that were broadly comparable in similar communities. This was a mixed methods study employing qualitative and quantitative methods: Quasi-experimental design using difference in differences analysis, CuCs Reading Assessment and Child and caregiver interview: socioeconomic status, home learning environment, schooling, and socioemotional learning. Group discussions and key informant interviews: Interviews with other stakeholders (education officials, coaches, and community learning facilitators) aimed at understanding program fidelity, relevance, acceptability, and scalability. The main instrument used was the Catch-up Clubs reading Assessment, designed by Save the Children and adapted from the ASER tool which has been widely used and validated. Supporting instruments such as child and caregiver interview, focus group discussions and key informant interviews were designed specifically for this evaluation. Data for the research was collected at baseline and three months later at endline. The evaluation used a cluster sampling methodology at school level to avoid spill-over which would compromise the internal validity of the design. Based on sampling calculations, providing a 95% confidence interval, a power of 80% and a minimum detectable effect of 30%, the evaluation sample was composed by 37 treatment schools and 37 control schools (21 from Gaibandha and 16 from Barishal), and 13 students per school per school for a total of 1020 students and 1020 caregivers. At baseline, a blanket survey was carried out testing for all students in treatment schools in grade 3 to 5, that had not participated on the previous round.
According to the endline findings, the treatment group improved two and a half more than control group. While the control group improved 13 percentage points between baseline and endline, the treatment group increased 32 percentage points. The impact of CuCs on reading assessments was 19 percentage points. The learning trajectory of the lower performing students analyses inequality in terms of reading outcomes. Lower performing students were defined as those that were assigned to Letter level at baseline. The findings show that the impact on literacy outcomes of lower performers was 26% higher than the average student (by 5 percentage points), while the effect of CuCs on reading outcomes was 19pp for all students, the effect for lower performers students was 24pp. In addition, an indirect effect of the Catch-up clubs was an improvement of children’s social and emotional learning. In fact, students in the treatment group showed a clear improvement over students in the control group. Students were interviewed at baseline and endline with 7 questions. In conclusion, the CuCs model proved to be extremely effective in the districts of Gaibandha and Barishal of Bangladesh, with improvements of 0.71SD in reading outcomes. Lastly, The CuCs innovation has proven significant progress on student’s learning outcomes and should therefore be prioritized as a data-proven success. By using a rigorous methodology such as a difference-in-differences estimation, we can assert that the improvement in learning outcomes were a result of the participation of students in the clubs.

Author

• Mohammad Sarwar Basher, Save the Children in Bangladesh