Beyond the Score: Using Digital NAEP Math Assessment Data to Support Teachers

• In Event: Deeper Insights Through NAEP Innovation

Abstract

This proposal explores the diagnostic potential of multi-source data (process data integrated with response data) from large-scale assessments (such as NAEP), to shed light on student knowledge gaps and learning skills in mathematics. The work aims to establish a theoretical and methodological link between assessment analytics and the established principles of learning science, recognizing that observed behaviors are open to multiple interpretations and are influenced by the student's content mastery and motivation. We employ a framework utilizing Human-Centered Explainable AI to analyze NAEP profiles with a teacher co-design, to contextualize student behaviors that could supplement evidence for teachers to better understand students’ learning states (e.g., math proficiency, motivation, persistence, organization and time management skills). This approach aims to demonstrate how multi-source data can generate data insights that could inform professional development and guide teachers in data use for targeted intervention.

Authors

• Hongwen Guo, Educational Testing Service

• Matthew S. Johnson, Educational Testing Service

• Jeremy Lee, ETS

• Luis Enrique Saldivia, ETS

• Michelle Worthington, ETS

• Kadriye Ercikan, Educational Testing Service