Using Large Language Model to Analyze Chemistry Undergraduate Students’ Self-Constructed Concept Maps

• In Event: AERA Poster Session Friday 7:45 am
  In Poster Session: ATL Poster Session 2: AI Integration and Outcomes in Postsecondary Settings

Abstract

This study investigates using generative artificial intelligence (AI), specifically GPT‑4o, in assessing self-constructed concept maps in undergraduate science education. Drawing on Assimilation Theory and Generative Learning Theory, we examine whether AI can approximate expert human grading by analyzing the accuracy and completeness of concept maps created by 257 students. A reproducible pipeline was developed using both hard and soft voting strategies to aggregate GPT‑4o outputs. Results indicate that hard voting outperformed soft voting in accuracy, precision, and F1 score, with the highest accuracy reaching 96.8% under a categorical scoring rubric. These findings demonstrate the feasibility of using AI to assess complex student-generated visual representations and highlight the potential of AI-integrated assessments to support scalable and meaningful feedback in educational contexts.

Authors

• Gan Jin, Washington State University

• Tingting Li, Washington State University

• Hyeonji Julia Lee, Washington State University

• Yu Xue, Washington State University

• Peng He, Washington State University

• Olusola Olalekan Adesope, Washington State University

• Chloe G. Dydasco, Washington State University

• Oluwafemi J. Sunday, Washington State University

• Krista Nishida, Washington State University