Effectiveness of NGO-led intervention on early grade reading through government system – Evidence from SERI program in India

• In Event: Best practices to achieve scale and sustainability in early grade reading - evidence from India

Proposal

Scaling up Early Reading Intervention (SERI) is a USAID-supported, large-scale and innovative intervention aiming to improve reading skills of early grade children in government primary schools in India. SERI intends to roll-out effective implementation models to improve reading skills by scaling NGO-led interventions through the government system. Based on a scaffolded approach, SERI is trying to create three type of implementation models: (i) Demonstration – demonstration of the effectiveness of the intervention through direct implementation in select government primary schools in some districts; (ii) Partnership – collaborate closely to assist the state governments to expand the intervention across entire district; and (iii) Scale-up – government taking on and replicating the program in other sub-provinces.

SERI demonstration model was first initiated at 180 schools in two states – Chhattisgarh and Uttarakhand – of India at the beginning of the 2015-16 academic year. In 2016-17, SERI partnership model was introduced at another 1,010 schools in these two states. In 2017-18, the program, both demonstration and partnership models, was expanded to new geographies (like Uttar Pradesh and Madhya Pradesh) in India.

To assess the effect on children’s reading skill, two separate impact evaluations for the demonstration and partnership models were conducted alongside the launch of SERI in Chhattisgarh and Uttarakhand. Following quasi-experimental design, impact under each model was estimated by the difference-in-difference methodology using Early Grade Reading Assessment (EGRA) data (Word Bank 2011; Kuhn, Schwanenflugel & Meisinger, 2010). Under each evaluation, data was collected from different cross-sections of children studying in a sub-set of intervention schools and same number of matching non-intervention schools at different points of time: (i) Baseline – a sample of start-of-Grade 1 children from both intervention and non-intervention schools was assessed before the launch of the program; and (ii) Endline – after two years, a different cross-section of end-of-Grade 2 children was assessed in the same schools.

Results from these evaluations revealed that children in SERI schools (under both demonstration and partnership models) performed better and experienced significantly greater two-year gains in reading skills than children from non-intervention schools. Effect sizes estimated for different reading assessment tasks were marginally higher for schools under demonstration model as compared to schools under partnership model. However, initial analysis showed that the impact in partnership school was achieved at much lower costs than the cost incurred to achieve almost similar results in demonstration schools. The panel intends to conduct more in-depth analyses to understand and compare the cost-effectiveness of the two implementation models of SERI.

These results provide evidence to advocate for supporting NGO-government collaboration to improve reading skills of early grade children at scale in resource-constrained environments. In this context, the panel would like to learn from the audience about their experience of such collaborative approaches in other geographies. The panel also feels that recurrent cost-effectiveness analyses in-built within the evaluation framework is a necessity for meaningful implementation of similar approaches in future.

Author

• Pinaki Joddar, Room to Read