Measuring the long run effects of early reading interventions through linking experimental, administrative and spatial data

• In Event: Digital possibilities for improving and evaluating early grade reading: Insights from a series of randomised experiments in South Africa

Proposal

In recent years, there has been growing support for improving the quality of foundational literacy and numeracy, especially in countries with weak educational outcomes. The motivation behind this support is that early interventions will be more cost-effective in the long run than later education interventions when children may already have fallen too far behind. The case for early interventions, therefore, rests on the notion that early learning interventions must generate long-term positive impacts on later educational outcomes. Although some famous studies have reported long-term impacts of having attended preschool, empirical evidence on the long-term impacts of foundational literacy and numeracy programmes is still limited. Most impact evaluations do not report impacts observed more than a month after the interventions ended.

South African children who benefited from the most impactful Early Grade Reading Study intervention when they were in grades 1-3 (2015-2017), have now been tracked to subsequent grades. Follow-up assessments of learning were conducted in grade 4 (2018) and grade 7 (2021). Moreover, through linking the EGRS data to administrative data at an individual level, we have been able to track progression of our original sample to grade 9 (2023). Recent improvements in South Africa’s Education Management Information System (EMIS) has made such analysis possible, and this illustrates the increasing potential for new technologies to improve education system planning and analysis.

This paper will report on the impacts of the early reading intervention on a range of long-term outcome measures including reading and literacy outcomes in Home Language and in English, as well as the impact on grade progression. Positive long-term impacts are observed across all of these three main groups of indicators. The results are discussed with reference to alternative theories for how early learning interventions might have long-term benefits. Some theories would emphasize how later skills build on earlier skills. On the other hand, the positive impact on grade progression, might be important for keeping more pupils at an appropriate age for their grade, something that we know is predictive of ultimately completing secondary school in contexts like South Africa.

Lastly, spatial data on the location of schools, is used to explore and illustrate further analytical possibilities when linking different sorts of administrative data to experimental data. Spatial analysis can be used to explore potential heterogeneities in the long-term effects of early educational interventions.

Authors

• Jonathan Stern, RTI International

• Matthew Jukes, RTI International

• Nompumelelo L Mohohlwane, Department of Basic Education

• Brahm Fleisch, University of the Witwatersrand

• STEPHEN G TAYLOR, Department of Basic Education