International impact benchmarking

• In Event: Ensuring quality and standards in AI-EdTech Part 2

Proposal

There is a significant gap between the widespread adoption of educational technologies (EdTech) in classrooms worldwide and the limited evidence of their actual impact on learning. While schools and policymakers have often based decisions on cost, accessibility and ease of use, and investors have prioritised scale and user acquisition, the question of whether EdTech improves learning outcomes has frequently been overlooked. This lack of evidence-based decision-making can undermine the quality of teaching and learning: educators may use tools that do not support instructional goals, students may experience reduced or even negative learning outcomes, and administrators may direct scarce resources toward solutions that are ineffective or unsustainable.
This presentation argues that evidence-based evaluation must become a central criterion for EdTech adoption. To support this shift, we propose a comprehensive evaluation framework that establishes a set of aspirational, research-informed quality indicators. The framework is structured around five dimensions: efficacy, effectiveness, ethics, equity, and environmental impact. The 5Es Framework incorporates multiple indicators designed to capture both quantifiable outcomes, such as test score gains, and broader educational considerations, such as inclusivity, sustainability, and student engagement.
The framework was developed through a structured, multi-stage process that synthesised findings from a wide body of research and expert consultation. Its purpose is to provide a clear, holistic set of measures that can guide decision-makers in evaluating EdTech products, while also being adaptable to different contexts and needs. Unlike traditional approaches that focus narrowly on single metrics, this framework recognises that education is a complex and embodied process, shaped by social, ethical, and environmental factors as much as by measurable academic outcomes.
This presentation’s contribution to the panel is threefold. First, it will be shared how the framework advances the evaluation of EdTech by offering a multidimensional model that enables comparability across tools and contexts. Second, ideas on how policymakers, school leaders, and investors can be supported with transparent, practical criteria for procurement and resource allocation. Finally, it supports a shift toward outcome-based contracting and evidence-driven investment and indirectly mobilises the EdTech developers to strengthen the quality and credibility of their products.

Author

• Natalia Kucirkova, University of Stavanger