Smartphone-enhanced structured pedagogy implementation: First results from the Tools for Foundational Learning Improvement impact evaluation

• In Event: From Constraints to Change: Data-Driven Approaches to Transform Early Literacy

Proposal

We present the initial results from a randomised controlled trial of Tools for Foundational Learning Improvement (TFLI). Developed by Inspiring Teachers, TFLI is a smartphone-enhanced structured pedagogy program where teachers are given upfront training on the science of reading, and equipped with high-quality, semi-scripted lesson plans linked with student workbooks, to run daily literacy lessons and given coaching. The program incorporates a digital layer: teachers and coaches assess children's literacy skills and are provided with integrated student tracking, coaching management tools, and training videos through a mobile app called SmartCoach. We study the effects of the program's literacy model, Inspiring Reading, on first-grade students in low-fee private schools in Cape Coast, Ghana. We randomly assigned 80 schools to either the Inspiring Reading program or a control group, and measured outcomes using end-of-year Early Grade Reading Assessments (EGRAs) and survey data.

The program causes large increases in student learning: our pre-specified primary outcome, overall EGRA reading scores, increases by 0.502SDs (p=0.015) which is the equivalent of more than two years of progress under the status quo. This puts TFLI at the 91st percentile of all reading interventions within the first year of what will be a 3-year program for children. The effects on individual components of the reading score are consistent with the program's theory of change: the largest effects are on mapping letters to sounds (d=0.757, p<0.001) and phonemic awareness (d=0.709, p<0.001). These are the key skills targeted by the program in grade 1. The impacts on oral reading fluency and reading comprehension are smaller and do not reach conventional levels of statistical significance, although they are quite large relative to typical impacts in the literature (d> 0.2). These are downstream skills that the program is building toward, and where effects are more likely as students progress through the program in grades 2-4. The smallest effects are on listening comprehension and letter names, which are targeted strongly by status quo teaching methods.

These impacts vary greatly across the distribution of test scores. In particular, there are large and statistically significant reductions in the fraction of students who cannot recognise words or read any words in a passage, both of which fall by over 40%. The effects also appear to be larger for male students; control-group girls are ahead by 0.26SDs, and the treatment closes 2/3 of this gender gap. This suggests that it may be more beneficial for weaker students more broadly. This would be consistent with the program's design, which focuses on supporting teachers in using assessment-informed instruction and in-classroom remediation. The benefits are larger in schools with female principals; we see no evidence of differences in effects by teacher gender or based on teacher-student gender match.

Our findings have important policy implications for improving foundational learning in low-income settings. The intervention is designed for scaling: It is designed to be low-cost and delivered through existing government workforces, and it is English-language-first, which challenges its adaptation to other contexts in Africa relative to mother-tongue-first interventions, which need to be re-developed for each new language. Moreover, the teacher guide designers employed a component-based design system and generative AI tools to accelerate lesson guide and workbook development, which could further accelerate adaptation to other national curricula. They are also using those same tools to help adapt the program for Uganda and exploring adaptation to Zambia. TFLI has the potential to make a major contribution to closing the learning gap between schools in Africa and those in the developed world.

Authors

• Jason T. Kerwin

• Monica P Lambon-Quayefio, University of Ghana

• Simon Graffy, Inspiring Teachers

• Erik T. J. Andersen, University of Washington