Examining Students’ Motivation and Knowledge Outcomes With Conversational AI in Math Learning Through an Experimental Study

• In Event: The Feasibility and Effectiveness of AI in Learning and Assessment

Abstract

This experimental study investigates the impact of conversational AI (ConvAI) on students’ academic motivation and conceptual change in algebra learning. A between-subjects design with 151 participants assessed the effectiveness of ConvAI in enhancing mastery approach, self-efficacy, and conceptual understanding compared to traditional methods. Results from two-way MANCOVA and ANCOVA analyses reveal significant increases in academic motivation within the ConvAI group, particularly in areas of mastery approach and self-efficacy. However, conceptual change did not significantly differ overall between groups, though it was moderated by individual differences in academic motivation. Our open-sourced ConvAI implementations and study findings provided support and implications for educational practitioners and researchers to design and develop pedagogically meaningful ConvAI in math learning.

Authors

• Chenglu Li, University of Utah

• Bailing Lyu, University of Utah

• Yukyeong Song, University of Tennessee

• Wanli Xing, University of Florida