Digital possibilities for improving and evaluating early grade reading: Insights from a series of randomised experiments in South Africa

Group Submission Type: Formal Panel Session

Proposal

Over the last ten years, the South African Department of Basic Education, working with several academic partners, has undertaken a series of studies aimed at building evidence about what works to improve the teaching and learning of reading at scale. A series of three randomised control trials were the core of this research programme, but a range of other research methods were included, leading to a rich variety of evidence and insights that have emerged. This panel does not report on the primary results of any of the randomised experiments, but presents insights based on what might be called the “second generation” of research questions emerging from the rich data that now exists.

The original Early Grade Reading Study (EGRS 1) was a randomised control trial conducted across 230 schools in the North West Province in 2015-2017. Three alternative interventions, all aimed at improving Home Language (Setswana) reading and literacy were assessed. The most impactful intervention was a structured learning programme that provided lesson plans, additional reading materials and on-site coaching to teachers. This led to a second Early Grade Reading Study (EGRS 2), implemented in the Mpumalanga Province between 2017 and 2019. This study compared on-site coaching to a virtual coaching intervention. A third study undertaken between 2021 and 2023, provided a similar combination of support to teachers but this time compared on-site coaching to an internal model where the school’s own Departmental Head was trained to become an internal coach.

This panel presents three distinct analyses flowing from this series of Early Grade Reading Studies. A focus of the panel will be on drawing out the areas in which digital technologies were used either in the design of early grade reading programmes or in evaluating their impact.

The first paper discusses a progression from the Early Grade Reading Study (EGRS 1) findings in 2015-2017, which demonstrated the effectiveness of on-site coaching, to subsequent investigations into virtual coaching modalities. Despite the appeal of scaling through digital means, results from EGRS 2 indicate that virtual coaching did not significantly enhance reading outcomes. This led to further explorations into decentralized coaching models, engaging Department Heads and Subject Advisors, yet these too struggled to match the efficacy of direct, in-person coaching.

The second paper shifts focus to the analytical potential of digital user data. By tracking the backend interactions of teachers with a digital application containing daily lesson plans, this paper unveils patterns in curriculum coverage. Notably, it reveals a decline in interaction frequency towards the end of weeks, terms, and the academic year, highlighting the challenges of ambitious curricula. This digital trace offers a novel lens through which to assess the fidelity of program implementation and its variance across different educational contexts.

The third paper caps the panel with a longitudinal perspective, tracing the cohort from the initial EGRS interventions up to Grade 9. By integrating experimental data with administrative records, this study not only shows the sustained benefits of early reading interventions, but also introduces spatial analytical techniques to examine geographical variations in impact. The paper discusses some of the channels through which early foundational literacy interventions might ultimately have positive impacts on later educational outcomes.

Together, these papers show the value of a coherent set of government-led studies, conducted over a sustained period of ten years, drawing on a range of research methodologies and data sources.

Sub Unit

• Global Literacy SIG

Organizer

• STEPHEN G TAYLOR, Department of Basic Education

Chair

• Brahm Fleisch, University of the Witwatersrand

Individual Presentations

• The impact of alternative technologies for delivering coaching to teachers - Brahm Fleisch, University of the Witwatersrand; Nompumelelo L Mohohlwane, Department of Basic Education; STEPHEN G TAYLOR, Department of Basic Education; Stella Banyana Mosimege, Department of Basic Education

• Using back-end app usage data to understand curriculum implementation and explain programme effectiveness - STEPHEN G TAYLOR, Department of Basic Education; Kolosa Nonkenge, Department of Basic Education; Nompumelelo L Mohohlwane, Department of Basic Education

• Measuring the long run effects of early reading interventions through linking experimental, administrative and spatial data - Jonathan Stern, RTI International; Matthew Jukes, RTI International; Nompumelelo L Mohohlwane, Department of Basic Education; Brahm Fleisch, University of the Witwatersrand; STEPHEN G TAYLOR, Department of Basic Education

Discussant

• Benjamin Piper, Bill & Melinda Gates Foundation