Designing Adaptive Dialogues in Inquiry Learning Environments to Promote Science Explanations (Poster 2)

• In Event: Automated Analysis of Students' Science Explanations: Techniques, Issues, and Challenges

Abstract

Purpose
Students need opportunities to reflect on, elaborate, distinguish among, and refine their ideas to develop an integrated understanding (Linn & Eylon, 2011). Teachers notice students’ sense-making and want to respond to the ideas of each student (Luna, 2018). Using state-of-the-art idea-detection natural language processing, our research practice partnership designed and tested a web-based, adaptive dialogue to engage students in analyzing their own ideas. Informed by the Knowledge Integration (KI) framework (Kali, 2006), the dialog engaged students in sharing their ideas, generating new ideas, and improving their explanations.

Methods
One 9th grade teacher and their 89 students completed the dialog in a unit on a Global Climate Change unit [website blinded] (dropped missing data; 66 students in analysis).
We analyzed data from the Car on a Cold Day problem (Table 4). A corpus of 1000+ student explanations for the ‘Car on a Cold Day' item was used to build the scoring rubric and to train an NLP model for detecting student ideas (Authors, 2022a). Prompts were designed for each detected idea, to engage the student in a conversation about their idea [Fig. 1]. The dialog was embedded in the middle of the unit.
We analyzed the automated KI scores and ideas detected for each student's initial explanation, responses to the dialog, and revised explanation. We used a revision rubric to categorize how students integrated ideas when revising (Authors, 2022b).

Results
The majority of students revised their initial explanation after the dialog (94%). This is a higher rate of revision compared to prior studies reporting under 50% of students engaging in sustained revision when an activity calls for it (Tansomboon et al., 2017).
The dialog elicited new ideas for each student. The 66 students expressed 70 new ideas in their revised explanations, 79% of which were normative. The adaptive prompt was more effective than the generic, eliciting 51 new ideas (86% normative) compared to the generic which elicited 21 new ideas (60% normative).
The majority of the students leveraged their new ideas from the dialog to improve their understanding. 52% integrated a new idea they expressed in the dialog into their revised explanation, and 17% added an idea from the dialog to the end of their initial explanation. 25% rephrased their initial explanation. Students significantly improved the KI level of their explanations from their initial to their revised explanation [Initial explanation KI score M=2.65, SD=.81; Revised explanation M=2.94, SD=.91; t(62)=3.2, p<.01].

Significance
The adaptive idea-detection dialog engaged the majority of students in revising and improving their science explanations. The majority of the students elaborated on the mechanism underlying their initial idea by incorporating a new idea elicited in the dialog - rather than accumulating disparate ideas (Linn & Eylon, 2011). Our next step is to design ways to further help students integrate the ideas elicited into an evidence-based argument and to distinguish among the ideas elicited to select the most productive to strengthen their explanation.

Authors

• Libby F. Gerard, University of California - Berkeley
  Presenting Author

• Allison Bradford, University of California - Berkeley
  Non-Presenting Author

• Kenneth Steimel, Educational Testing Service
  Non-Presenting Author

• Brian Riordan, Cisco
  Non-Presenting Author