A Meta-Review of AI-Powered Chatbots and Their Impact on Learning Outcomes

• In Event: AI-Enhanced Student Learning Outcomes

Abstract

This meta-review synthesized 12 meta-analyses and estimated the weighted effect size of AI-powered chatbot interventions on student learning outcomes in formal education settings. The results showed a large average effect size for overall learning (g = 0.74), a moderate-to-large effect for cognitive learning (g = 0.67), and a moderate effect for affective learning (g = 0.61). Substantial heterogeneity was observed across studies (I² = 83.5% for overall learning, 84.23% for cognitive learning, and 76.38% for affective learning). Additionally, we identified issues related to transparency of reporting in the included meta-analyses, including a lack of rationale for selecting random- vs. fixed-effect models and incomplete reporting of sample sizes.

Authors

• Secil Caskurlu, Florida State University

• Huan Kuang, Florida State University

• Seungjie Clarice Han, Florida State University