Integrating digital personalised learning into pre-primary education systems to improve literacy and numeracy outcomes: a randomised controlled trial in Kenya

• In Event: Designing for evidence-informed digital personalised learning: facilitating critical comparisons, understanding limitations, and establishing future directions

Proposal

CONTEXT AND RELEVANCE:

The potential of Digital Personalised Learning (DPL) is indicated by the first meta-analysis of DPL in low- and middle-income countries (LMICs), revealing a statistically significant positive effect on numeracy and literacy, with a combined effect size of 0.18 from 15 studies involving over 50,000 learners ([AUTHOR], 2021). However, this and related research (e.g., Van Schoors et al., 2021) highlights a significant gap: the role of teachers in implementing DPL is limited in LMICs, with existing research focused on ‘supplementary’ use of DPL that is not closely aligned with teachers’ classroom practice. Typically, this involves additional technology sessions detached from regular instruction – for instance, as a direct substitute for traditional teaching (Bettinger, Fairlie, Kapuza, Kardanova, Loyalka et al., 2023), or as part of an after-school programme (Muralidharan, Singh & Ganimian, 2019). Such findings emphasise the need for evidence on the impact of ‘classroom-integrated’ DPL implemented in combination with the curriculum and teacher practices.

RESEARCH FOCUS AND DESIGN:

The present study addresses this gap by investigating a DPL programme closely aligned with a national curriculum and pedagogy for the first time. Personalised to each learner’s device interaction history, the [ANON] DPL tool optimises the content sequencing of 348 curriculum-aligned learning units to support pre-primary numeracy and literacy. Teachers further support personalisation based on their pedagogical judgement and the tool’s alignment of DPL content with digitised lesson plans.

Building on extensive collaborative design-based research involving teachers, technology developers, researchers, and stakeholders ([AUTHOR], 2024), a randomised controlled trial (RCT) evaluating the impact of a classroom-integrated DPL programme in Kenya is reported. This research involved 1955 learners, aged 4 to 6 years old, across 291 government schools. Numeracy and literacy skills were assessed at baseline, midline, and endline over 13 months using the International Development and Early Learning Assessment (during 2022-2023).

FINDINGS:

Findings demonstrate a substantial effect of 0.534 SD on overall scores from baseline to endline, with similar impacts observed for numeracy (0.450 SD) and literacy (0.449 SD). Subgroup analyses examined two equity dimensions (baseline achievement levels and gender) and one structural factor (class size). The intervention proved equally effective for both boys and girls and across different class sizes, but was more effective for low compared to high-achieving learners in numeracy. Notably, a reduction in the extent of effectiveness was observed following midline, warranting further investigation.

CONTRIBUTION AND SIGNIFICANCE:

This paper offers a critical discussion of the impact of a DPL tool which is integrated into Kenyan pre-primary classrooms, aligned with curriculum and pedagogy. While questions remain about the potential reduction in the extent of effectiveness, the implemented approach demonstrates promise. Discussion of these results highlights the importance of viewing DPL as a multi-layered construct, especially when considering generalisability of the findings. Recommendations are also given for interpreting DPL impact studies, considering the vast range of tools classified under the broad DPL ‘umbrella’.

Author

• Louis Major, The University of Manchester