The impact of NEGRP, findings from an external evaluation

• In Event: Learning Lessons from Positive Cases: Examining Enabling Conditions and Supports for Early-Grade Reading in Nepal

Proposal

This paper will share endline results showing the impact of the Nepal Early Grade Reading Program (NEGRP). The NEGRP aims to improve reading skills of students in Grades 1-3. All students (regardless of mother tongue) received the NEGRP package, which includes distribution of Nepali Teaching and Learning Materials (TLM), in-service teacher training on the use of TLMs, head teacher and school management committee (SMC) member orientation, teacher coaching, mentoring, and support, parent and community level engagement activities, and public service announcements on the radio and newspapers to promote early grade reading in the community.

The evaluation measures the effect of NEGRP on reading outcomes of native (L1) and non-native (L2) Nepali speakers, the extent to which NEGRP changed teachers’ reading instructional practices in the classroom, and changes in school management support for EGR.
The NEGRP was not implemented randomly. Thus, we used a quasi-experimental approach, which serves as a rigorous alternative to a randomized evaluation, and allows for the credible estimation of program impacts. We matched comparison and treatment schools, selecting schools from control districts that are most similar to schools in treatment districts and then estimated the program impact using a Difference-in-Difference approach.
At endline, the EGRP had positive effects on all EGRA measured reading skills for both L1 and L2 learners and across grades. Reading performance has increased for treatment learners however, there is still room for improvement. At endline, most grade 1 and around a quarter of grade 3 learners are still not able to read a single word from a connected paragraph.

Both L1 and L2 learners benefited from the program. However, there is a large gap between L1 and L2 learners’ reading skills. The gap is approximately the equivalent of one full year of schooling. NEGRP was able to improve performance among L2 learners but not enough to reduce the disadvantage they experience as non-Nepali speakers.

Examining the channels through which the program has functioned, there is no evidence that the program has led to changes in parents’ at-home support for their children’s reading development, which seems quite high already for all groups. School Management Committee support for reading activities shows a very modest improvement only in some districts.
The program has been quite successful at ensuring access to materials, including students’ access to Nepali-language workbooks, and additional children’s reading materials, and teachers’ access to teaching guidelines, materials, and curriculum. Almost all teachers report using these resources.
There is evidence that the program has had a positive effect on teachers’ reading instruction, as captured by the classrooms observation exercise. The percentage of teachers conducting desirable reading instruction activities in class has increased in treatment schools. Support supervision of teachers is still not universal. Although treatment teachers have higher probability of receiving support, there is still a significant fraction of teacher that report no supervision at all.

It is likely that the positive effects of the program have functioned via a combination of improved teaching practices with broad access and use of learning and teaching materials.

Author

• Alicia S. Menendez, University of Chicago & NORC