PME-NA

Expert Analysis of AI-Generated Theoretical Analyses in Mathematics Education

• In Event: Expert Analysis of AI-Generated Theoretical Analyses in Mathematics Education

Description

This study investigates an AI system’s strengths and limitations for analyzing theoretical issues in mathematics education. Using a comprehensive exam question about Mathematical Knowledge for Teaching (MKT), we explored how AI, specifically Google's NotebookLM, interprets and distinguishes between Ball’s and Thompson's MKT frameworks. Faculty with publications in MKT research reviewed and identified both surprising competencies and subtle limitations in the AI's theoretical analysis. We present a framework categorizing these limitations as field knowledge, attribution, argumentation, illustration, and artificial neutrality. Our findings suggest that current AI systems can do useful theoretical analysis in math education at a graduate level but demands alertness to nuanced failings which often require expert knowledge to detect. Implications for research and doctoral pedagogy are advanced. The study offers insight into both current AI capabilities and critical evaluation methods for AI-generated academic content.

Authors

• Jennifer Kleiman, University of Georgia

• Cameron Byerley, Colorado State University

• Kevin C. Moore, University of Georgia