Can ChatGPT Help Analyze Collaborative Mathematical Argumentation? Comparing AI Tools for Classroom Argumentation Analysis

• In Event: Examining the Intersectionalities of Mathematics and Languages

Abstract

Collaborative argumentation is central to the practice of mathematics, yet analyzing small-group classroom discourse remains challenging and time/resource-intensive for educators and researchers. This study explored whether commonly available large language models (LLMs) like ChatGPT could support classroom discourse analysis of the Conversational Argument Move AnaLysis (CAMAL) dataset of 165 transcripts, comparing LLMs to purpose-trained neural networks against human coding. Results showed comparable performance from some but not all LLMs, with the best LLMs strong at identifying explicit reasoning but weaker with implicit social aspects. Findings suggest automated tools could help mathematics education researchers analyze classroom discourse at scale, enabling systematic study of mathematical reasoning development while supporting teachers in assessing argumentation.

Authors

• Jennifer Kleiman, University of Georgia

• Yizhu Gao, University of Georgia

• Xiaoming Zhai, University of Georgia