BackHome
Refresh: Off
Paper Summary
Share...

Direct link:

AI-Supported Feedback to Scaffold Scientific Argumentation

Sat, April 11, 3:45 to 5:15pm PDT (3:45 to 5:15pm PDT), Los Angeles Convention Center, Floor: Level Two, Room 515B

Abstract

Objective
The Automated Measure of Student Reasoning Patterns in Science (Auto-SPIN) project aims to develop and validate an AI-supported classroom assessment tool that measures middle school students’ reasoning patterns during scientific argumentation about ecosystem phenomena. Research has shown great potential using AI to automatically grade student arguments (Wilson et al., 2024; Author, 2023) but providing feedback to facilitate effective learning is challenging. As part of the broader research effort, this study reports findings from a cognitive lab, examining how students interact with AI-generated feedback and revise their arguments.

Perspectives
This work is grounded in scientific argumentation theory of instructionally supportive feedback. Drawing on the Claim-Evidence-Reasoning (CER) framework (McNeill & Krejcik, 2008), the project defines high-quality arguments as those that coherently link claims and relevant evidence through scientifically grounded reasoning. The feedback system draws from research on scaffolding (e.g., Reiser, 2004) and reflects an epistemic agent perspective (Gonzelez-Howard & McNeill, 2020), encouraging students to view themselves as knowledge builders who evaluate the validity of their own reasoning (Stroupe, 2014). Rather than just identifying correct answers or missing components of an argument, the feedback is designed to prompt reflective questions like “Is my argument valid?,” “How does my reasoning hold up to critique?”

Methods
This study reports findings from cognitive lab sessions with 30 students in Grades 6-8. Each student interacted with an AI-supported task focusing on a scenario of rootworm invasion in a school garden. Students ran online simulations, developed scientific arguments, received AI feedback, and revised their arguments. Data sources included screen recordings, concurrent think-aloud protocols, written task responses and log data with timestamps, in addition to post-task interviews. All sessions were recorded, and transcripts were coded to analyze how students interpreted and responded to feedback. The analysis focused on changes in argument completeness and coherence, and types of revisions and perceived value of the feedback.

Results
Although data collection is ongoing, preliminary analysis of 23 sessions indicates that almost all students revised their responses after receiving feedback. However, the depth of engagement varied. Some students made surface-level changes (e.g., rephrasing), while others restructured their arguments, replaced non-supportive evidence, or added scientific reasoning. Notably, some students interpreted the feedback as a signal that they had done something wrong, rather than a tool to deepen their thinking or explore alternative ideas. This corrective interpretation limited some students’ engagement with feedback and highlights the importance of how feedback is framed in supporting reflection and further exploration. Additional findings from the reasoning pattern analysis, which is still underway, will be reported at the presentation.

Significance
This study contributes to emerging research on the role of AI in formative assessment (e.g., Author, 2024a; Author, 2024b). Auto-SPIN illustrates how AI can serve as a scalable instructional partner for teachers and students by providing timely and targeted support for scientific reasoning. The findings offer design insights into how AI feedback systems can be aligned with disciplinary goals while being responsive to individual student needs. This work has implications for the future of AI-enhanced instruction and assessment.

Authors