Analyzing High School Students' Code Comprehension and Strategies Using SOLO Taxonomy and Large Language Models

• In Event: AERA Poster Session Saturday 1:45 pm
  In Poster Session: Computer Science Education approaches and techniques in context

Abstract

Understanding students’ programming knowledge is essential for providing effective, tailored scaffolding. This study examines high school students’ code comprehension across block-based and text-based programming tasks using think-aloud data analyzed through the SOLO taxonomy framework. Twenty-seven students completed Scratch and Python tasks focused on loops and conditionals. SOLO levels and code comprehension strategies were identified from student reasoning. The study also evaluated large language models for automating SOLO classification. Results showed higher SOLO levels in block-based tasks and a performance drop in text-based tasks, with strategy use differing by modality. While LLMs accurately classified higher-level responses, they struggled with nuanced reasoning (Prestructural levels), highlighting the need for human oversight. Findings offer theoretical and practical insights into scalable assessment and instructional design.

Authors

• Shan Zhang, University of Florida

• Priyadharshini Ganapathy Prasad, University of Florida

• Toni Earle-Randell, University of Florida

• Zifeng Liu, University of Florida

• Yang Shi, Utah State University

• Suma Bhat, University of Illinois at Urbana-Champaign

• Maya Israel, University of Florida