Addressing the Global Learning Crisis through Evaluation of Educational Technology

• In Event: Data, Technology, and Talent in Education Systems: Policy Pathways to Address Global Learning

Proposal

Background
We are facing a global learning crisis. Over 55% of children worldwide do not have the basic reading, writing, and numeracy skills needed to succeed in everyday life (UNESCO, 2017). This lack of foundational learning undermines children’s lifelong opportunities and countries’ social and economic development (Lu et al., 2016). The COVID-19 pandemic has exacerbated these challenges by disproportionately affecting the poorest and most vulnerable children, thereby deepening pre-existing inequalities and limiting progress towards Sustainable Development Goal (SDG) 4, which aims to ensure inclusive and equitable quality education for all (United Nations, 2021).
Educational technology (EdTech) has been promoted as a promising pathway for improving learning outcomes at scale and for reaching marginalised populations. A growing body of evidence suggests that when designed and implemented effectively, EdTech can support personalised learning, enhance teaching quality, and address inequities in access to high-quality education (Outhwaite et al., 2023; Pitchford, 2023). However, the evidence base remains fragmented, particularly about how EdTech can be most effectively implemented and evaluated across diverse contexts, especially in under-resourced educational systems, such as those in low- and middle-income countries (LMICs).
This study seeks to contribute to this evidence base through the evaluation of seven EdTech tools supported by UNICEF’s Blue Unicorn Portfolio across six LMICs. Each tool is designed to enhance either foundational literacy or numeracy skills, or to strengthen teacher competencies and learning opportunities for children with special educational needs.
Theoretical Framework
This study adopts a structured and evidence-informed evaluation framework designed to rigorously assess the effectiveness of the selected tools in addressing the global learning crisis. This approach emphasizes measurable outcomes in foundational literacy and numeracy, aligning with international benchmarks for early learning. By employing a portfolio approach, we aim to evaluate all seven EdTech tools across multiple contexts and use cases, allowing for comparative insights as well as detailed insights into how individual tools work in individual geographies. This methodology was chosen to maximize both the validity of the findings and their relevance for scaling and policy decision-making. A theory of change has been developed for each tool by the research team to better understand their individual assumptions, processes, and mechanisms for supporting learning.
Methodology
This study aims to evaluate the EdTech tools for improving foundational learning outcomes and teacher competencies in LMICs. To do so, this study adopts a quasi-experimental pre/post-test design to evaluate the impact of each EdTech tool in their specific country context, as well as the overall portfolio approach. Although not a randomized controlled trial (RCT), the design incorporates methodological rigour through purposive school selection and techniques for matching intervention and control schools. This strengthens the internal validity while accommodating real-world implementation constraints.
The intervention period is 12 weeks, amounting to 40–60 hours of engagement per learner, consistent with providers’ recommendations for optimal use. Variations in dosage across tools or contexts are systematically documented to account for any implementation differences.
The target population is learners in lower primary years (e.g., Grade 2). For each tool, a sample of around 600 children across intervention and control schools have been recruited from schools stratified across urban, peri-urban, and rural contexts to capture contextual variation.
Primary outcomes include children’s progress on literacy and numeracy scores, which are assessed using an adapted version of the EGRA and EGMA assessments, respectively. Secondary outcomes include a bespoke teacher survey, which captures teacher knowledge, practices, and competencies relevant to integrating EdTech in classroom instruction.
Planned Analyses
Due to the practical constraints of this study, the planned analyses will follow a pupil-level approach. For example, pre- and post-test comparisons will be used to assess changes in children and teachers’ learning outcomes between intervention and control groups. Effect sizes will be estimated to quantify the magnitude of impacts per tool in each country and afford cross-country and cross-tool comparisons. In addition, implementation data (e.g., fidelity, dosage, and teacher engagement) will be used to contextualise findings.
Theoretical Significance
This study contributes original insights into the role of EdTech in addressing the global learning crisis. By combining rigorous evaluation with real-world implementation, it addresses gaps in the literature concerning how personalised learning models work in practice, and how digital tools can be adapted across diverse contexts. The use of a portfolio approach enables both cross-context comparison and insights into individual tools’ impact. This design allows the project to interrogate not only whether EdTech works, but how, for whom, and under what conditions it is most effective.
Educational Significance
This study will also provide actionable evidence for educational practitioners, governments, NGOs, international agencies and other stakeholders striving to achieve SDG 4. Our findings will illustrate how scientific best practices can be combined with contextual realities in under-resourced settings to offer models for how large-scale EdTech interventions can be evaluated, adapted, and scaled.
By providing comparative data across six countries and seven tools, the study offers one of the most comprehensive real-world evaluations of EdTech in foundational learning to date. It thus addresses urgent questions about scalability, sustainability, and the role of digital solutions in mitigating learning losses, which have been intensified by the COVID-19 pandemic.
Conclusion
The global learning crisis demands urgent and evidence-based responses. This study represents a significant step toward understanding the potential and limits of EdTech in supporting foundational literacy and numeracy through robust quasi-experimental cross-country comparisons.
References
Lu, C., Black, M. M., & Richter, L. M. (2016). Risk of poor development in young children in low-income and middle-income countries: an estimation and analysis at the global, regional, and country level. The Lancet Global Health, 4(12), e916-e922.
Outhwaite, L., Ang, L., Herbert, E., Sumner, E., & Van Herwegen, J. (2023). Technology and learning for early childhood and primary education. UNESCO Global Education Monitoring Report.
Pitchford, N. J. (2023). Customised E-Learning Platforms. In T. Madon, A. J. Gadgil, R. Anderson, L. Casaburi, K. Lee, & A. Rezaee (Eds.), Introduction to Development Engineering. London: Springer
UNESCO. (2017). More Than One-Half of Children and Adolescents Are Not Learning Worldwide. Available from: http://uis.unesco.org/sites/default/files/documents/fs46-more-than-half-children-not-learning-en-2017.pdf
United Nations. (2021). The Sustainable Development Goals Report. Available from: https://unstats.un.org/sdgs/report/2021/The-Sustainable-Development-Goals-Report-2021.pdf

Authors

• Nicola Pitchford

• Natalia Kucirkova, University of Stavanger

• Jimmie Odhiambo, University of Wisconsin Madison

• Laura Outhwaite, International Centre For EdTech Impact

• Frank van Cappelle, UNICEF

• Rachel Foster, International Centre For EdTech Impact