Using AI-Based Formative Assessment and Scaffolding to Support Students’ Mathematical Practices Needed for Science (Poster 2)

• In Event: AI and Formative Assessment: The Train Has Left the Station

Abstract

Objectives
The goal of the current study was to analyze student data from two virtual science labs that included mathematical modeling, as per NGSS, in [ITS]. The labs utilize AI-based, real time formative assessments and scaffolds targeting the sub practices of mathematical modeling. This fine-grained formative assessment and scaffolding approach allow us to better understand precisely where students struggled most when engaging in mathematical practices during science inquiry and examine how well scaffolding supported students on a subsequent lab.
Theoretical framework
Science instruction in most of the US is guided by the Next Generation Science Standards (NGSS, 2013). The standards describe the authentic practices students need to engage in to understand phenomena. Mathematical modeling, an NGSS practice that includes skills such as plotting data (Beichner, 1994) and assigning functional relationships to graphs (Casey, 2015), are required in science, especially in high school and beyond. However, tudents often struggle with mathematical modeling in science contexts (Potgieter et al., 2008; Lai et al., 2016). [ITS] addresses these challenges via automatic assessment and scaffolding driven by AI algorithms to support students in real time during inquiry. These scaffolds are executed based on assessments of operationalized sub-practices. In the current study, we addressed: 1) which mathematical sub-practices students needed support on, and 2) if receiving real time scaffolds helped students improve their performance across these sub-practices (i.e., competency scores) in lab 2.
Methods
Participants were 70 eighth grade students in four science classrooms taught by one teacher in Northeastern US. Participants completed two [ITS] virtual labs on Forces and Motion (NGSS DCI PS2.A), each of which included four stages (Hypothesizing, Collecting Data, Plotting Data, & Building Models). Students’ competencies on each stage were formatively assessed, including its respective sub-practices. The two mathematical modeling stages, Plotting Data and Building Models, were the focus of this study (Table 1). Student log data on the types of scaffolds triggered in lab 1, associated sub-practices needing support, and competency scores from the two autoscored math modeling stages in both labs were analyzed.
----------------------------------------------Table 1---------------------------------------------------------
Results
Students required support on across all sub-practices in the Plotting Data stage and the Building Models stage. After receiving targeted scaffolding in the Plotting Data stage, students improved from the first to second lab following 3 of the 4 targeted scaffolds that addressed challenges across sub-practices in this stage. Students in the Building Models stage improved from the first to second lab following 2 of the 4 targeted scaffolds that addressed challenges across sub-practices in this stage. This suggests that most scaffolds provided sufficient support for these difficult sub-practices, as evidenced by improvement from lab 1 to 2.
Scholarly Significance
Our data suggest that AI based formative assessments and scaffolds, operationalized at the sub-practice level, helped students improve their math competencies in a rich science context. For the sub-practices on which there was no improvement after scaffolding, we will address whether changes to our scaffolds are needed or whether these specific sub-practices require multiple scaffolds over time to yield improvement.

Authors

• Ellie R. Segan, Rutgers University

• Janice D Gobert, Rutgers University

• Michael Sao Pedro, Apprendis