Are We Overestimating the Power of Generative AI? A Meta-Analysis of Its Impact on Student Learning

• In Event: Teaching and Learning with GenAI

Abstract

Generative artificial intelligence (GenAI) is rapidly being integrated into education, yet its true impact on learning outcomes remains unclear. This meta-analysis synthesizes findings from 56 studies (encompassing 91 independent effect sizes, N=5,215) to systematically evaluate the overall effects of GenAI on student learning. Results show a significant positive overall impact (g=0.692), with large gains in academic performance, critical thinking, and reduced anxiety. Effects were moderate for motivation, engagement, and creativity, but non-significant for self-efficacy. Moderator analysis indicates that GenAI's effects are more pronounced in higher education and in interventions lasting ten weeks or longer. These findings confirm the educational potential of GenAI while highlighting the need for thoughtful pedagogical design tailored to different contexts and learning dimensions.

Authors

• Yijia Yuan, University of Cambridge

• Weichao Si, University of Hong Kong