The Role of Teachers in Implementing and Enhancing Digital Personalized Learning: evidence from a multi-strand study in Kenya

• In Event: Enabling teachers to harness digital personalized learning to accelerate learning outcomes

Proposal

Since 2022, [author], in collaboration with researchers at [partner] and [partner], has been investigating the impact of a curriculum-aligned DPL tool on early-grade learning outcomes in Kenya, and - critically - the role of teachers in its implementation. The DPL tool, designed by [implementer], comprises an application installed on low-cost Android smartphones, which has over 200,000 active learners daily. Personalized to each learners’ device interaction history, this tool optimizes content sequencing of 348 curriculum-aligned learning units to maximize learner engagement, while teachers can further support personalization based on their pedagogical judgement, data dashboards and the tool’s alignment of DPL content with digitized lesson plans.

This multi-strand study has involved a range of methods to explore the impact of the [implementer] DPL tool and the different factors which affect its implementation. First, design-based research, involving 200 education stakeholders and co-learning activities with teachers, involved developing and evaluating different approaches of integrating the DPL tool into Kenyan classrooms. Second, two randomized controlled trials (RCTs), each involving c.300 schools, have assessed the impact of the tool on literacy and numeracy outcomes for pre-primary and early-grade primary learners, using the IDELA and EGRA/EGMA assessment tools respectively. Third, A/B/n testing, a controlled experimental approach which randomly assigns participants to different software versions, has assessed the comparative effectiveness of various design features and algorithm approaches in over 5,000 schools, collecting usage and learning data on the tool over a whole school term.

Results reveal the overall effectiveness of this classroom-integrated DPL tool for early-years learning (0.534 SD). However, they also highlight that ‘personalization’ does not happen in a digital vacuum, as teachers’ role in implementing the DPL tool has an impact on multiple areas:

Personalizing what DPL content is selected: A/B/n testing suggested that giving teachers choice to override the system-generated content selection, by applying their pedagogical judgement to select an alternative learning strand, can significantly improve learners’ digital formative assessment scores (p < 0.001). This suggests the potential of teacher-AI collaboration.

Personalizing who receives the DPL tool: the design-based research identified that learners whom teachers perceive to be “fast learners” – those who complete classroom activities first – are more likely to be given access to the DPL tool than “time-takers”. This suggests that teachers’ implementation of the tool in classrooms can impact on learners’ equality of access to DPL.

Personalizing pedagogical practice: A/B/n testing found a significantly higher impact when learner skill-level data was presented to teachers in a grouped format which reflects terminology used in the national curriculum (compared to listing all individual learners or not providing any data dashboard, p < 0.001). This suggests that viewing certain formats of data dashboards can prompt changes to pedagogical practice.

This paper offers a critical discussion of these findings, arguing that a nuanced approach to designing and implementing DPL, which recognizes the multi-layered role of the teacher in facilitating digital and pedagogical personalization, is needed.

Author

• Rebecca Daltry, EdTech Hub