Advanced Methodologies for NAEP Operations and Analyses (Table 15)

• In Event: AERA Roundtable Session Saturday 1:30 pm Four Seasons Ballroom 2-3

Session Type: Roundtable Session

Abstract

The session showcases five studies that employ advanced methodologies to enhance NAEP operations and inferences. The first paper attempts to optimize NAEP's reading router using Multi-Stage Adaptive Testing, increasing fairness and accuracy for low-ability students. The second and third papers leverage state-of-the-art artificial intelligence techniques to explore AI-assisted education in mathematics and to analyze teachers' open-ended responses, respectively. The last two papers focus on applying advanced methods to make inferences using NAEP data. The fourth paper applies ordinal logistic regression to examine how Black teachers in predominantly Black schools integrate civics-focused instructional techniques. The fifth paper uses hierarchical linear modeling to investigate the effects of school services and civic dispositions on student outcomes, highlighted the need for personalized educational approaches.

Sub Unit

• SIG-NAEP Studies

Chair

• Amy R. Dresher, Educational Testing Service

Papers

• Evaluating ChatGPT-4 and ChatGPT-4o: Performance Insights From NAEP Mathematics Problem Solving - Xin Wei, Digital Promise

• Chances Are? Teacher-Student Racial Matching as a Predictor for Enhancing Student Civic Opportunities and Civic Performance Outcomes - Chrystal S. Johnson, Purdue University; Chenchen Lu, Purdue University

• The Effects of Civic Disposition and Supporting Service on Civic Achievement: A Multilevel Analysis of NAEP Civics Data - Chenchen Lu, Purdue University; Shuangting Li, Purdue University; Chrystal S. Johnson, Purdue University