Leveraging a Technology-Driven Structured Pedagogy Programme to Transform Education: Evidence from Government Schools in Rwanda

• In Event: Education improvements: interventions and impacts

Proposal

Leveraging a Technology-Driven Structured Pedagogy Programme to Transform Education: Evidence from Government Schools in Rwanda

Background

Before the launch of the RwandaEQUIP programme, Primary school pupils in Rwanda demonstrated weak foundational literacy and numeracy skills. More than 9 in 10 pupils in a typical Primary 1 class were unable to read a single word in English, more than half unable to read a single word in Kinyarwanda, and the average pupil could not correctly solve more than 3 simple addition problems in one minute. In February 2022, the RwandaEQUIP programme was launched, providing 17 weeks of instruction in the second and third terms of the 2021-22 school year, and a full 30 weeks of instruction during the 2022-23 school year. The programme is built on the principles of structured pedagogy – a coordinated instructional improvement approach that combines multiple inputs, such as lesson plans for teachers, pupil textbooks, teacher training, etc., that work together to improve teaching and learning in every classroom. Importantly, the programme relies heavily on technology for the distribution of instructional content for pupils and guidance for teachers, the monitoring of learning outcomes, and the management of professional accountability among all stakeholders. After a net 47 weeks of programme delivery, the programme’s impact on educational outcomes was measured using a difference-in-differences approach and a longitudinal follow-up.

Research Questions

Can data-informed management and structured pedagogy improve learning?
Can it improve learning for pupils across a wide range of readiness levels?

Methods and Data

Data from 60 public Primary schools across Rwanda were used; of these, 30 schools were part of the RwandaEQUIP programme (“treatment” schools) and 30 were not (“comparison” schools”). From each school, approximately 15 pupils per grade in Nursery 3 through Primary 3 were randomly selected and assessed on their literacy and numeracy skills. English oral reading fluency and reading comprehension were assessed using passages from the Early Grade Reading Assessment (EGRA), and Kinyarwanda fluency and comprehension using passages from Local Early Grade Reading Assessment (LEGRA). Pupils’ numeracy skills were assessed using the Early Grade Mathematics Assessment (EGMA). In addition to pupil learning outcomes, teachers’ pedagogical practices were assessed using an adapted version of the TEACH tool (World Bank, 2022).

Programme impact during the first 17 weeks was calculated using the difference-in-differences method. The progress seen in comparison schools during this 10-week period established the growth trajectory expected in schools that operated in the “status quo” mode. The progress made in treatment schools during the same 17-week period was also measured, then the difference was calculated between the actual performance of pupils in treatment schools and their expected performance had they followed a status quo trajectory – yielding a quantitative measure of programme impact.

After 47 weeks of programme implementation, continued progress was measured through a longitudinal follow-up. In the 30 schools that comprised the treatment group, learning levels measured at the end of the 2022-23 school year were compared to those measured at the end of the previous school year (2021-22) as well as those projected for the end of the school year before the programme launched (2020-21).

In addition to calculating programme impact, other indicators of programme implementation were monitored over the course of the 47 weeks, including teacher attendance and time on task.

Preliminary Findings

Implementation of RwandaEQUIP, a comprehensive education programme that integrates technology into all elements of its structured pedagogy package, is producing measurable improvements in teaching and learning. Positive leading indicators emerged after 17 instructional weeks; reading fluency among RwandaEQUIP pupils improved at a rate 64% faster than among comparison pupils, and there were significant improvements in the quality and quantity of instruction, with teachers being present and on-task more than they were before the programme, and employing more effective pedagogical strategies. After 47 instructional weeks, dramatic improvements were seen across educational outcomes. In foundational literacy, RwandaEQUIP pupils increased their English reading fluency by twice as much as comparison pupils, and their reading comprehension at three times the rate of comparison pupils. In foundational numeracy, RwandaEQUIP pupils experienced an 18 percentage point increase in EGMA scores – which is double the 9 percentage point improvement in comparison schools.

Importantly, the RwandaEQUIP programme is effectively supporting pupils across a wide range of performance levels, rather than simply targeting a small number of high-performing outliers. At schools that joined the RwandaEQUIP programme at its launch, the overall distribution of learning levels was shifted in a positive direction as compared to comparison schools. The programme maintained gender parity in learning outcomes, and pupils achieved significant growth in learning under teachers with varying levels of English fluency.
We also find that the increased teacher accountability enabled by the RwandaEQUIP ecosystem was accompanied by a marked decrease in teacher absenteeism across the system (34% to 11%). The consistent presence of teachers in the classroom, combined with the fact that they were delivering high-quality lesson content using research-based pedagogical practices, and with a simultaneous increase in pupil attendance, translates to each pupil receiving an additional 18 hours of high-quality instruction each week.

Together, these results demonstrate that technology-enabled instruction and management can have positive impacts at multiple levels of an educational system – raising pupil learning levels, improving the quality of instruction, and increasing educator professionalism.

Contribution

We provide evidence that a comprehensive education programme that leverages technology for all aspects of programme implementation can deliver large improvements in learning outcomes on a national scale. We also provide evidence that the programme can improve learning outcomes for pupils across a wide range of baseline proficiency levels, thus producing more equitable outcomes.

References
World Bank. (2022). Teach Primary: Helping Countries Track and Improve Teaching Quality. https://www.worldbank.org/en/topic/education/brief/teach-helping-countries-track-and-improve-teaching-quality

Authors

• Priscilla Lu

• Keuna Cho, NewGlobe Education

• Pranav Bhargava, NewGlobe Education

• Sylvester Mchihi, NewGlobe Education