Paper Summary
Share...

Direct link:

Evaluating Conversation Patterns to Elicit Evidence of Students' Science Reasoning Skills

Sat, April 18, 10:35am to 12:05pm, Sheraton, Floor: Ballroom Level, Sheraton IV

Abstract

Prior research has investigated and identified many conversational patterns within tutorial interactions that improve student learning (Craig, Sullins, Witherspoon, & Gholson, 2006; Graesser & Person, 1994). These patterns typically involve deeper levels of cognitive processing and/or exchanging information with collaborative peers. Although these patterns of conversation have been studied in the learning domain, less is known about their potential for assessment purposes. The current work explores various conversation patterns, the evidence each pattern elicits from students, and how conversations may be implemented to gather rich information about students’ knowledge, skills, and interactions.
Specifically, a set of conversational patterns have been implemented within an adaptive scenario-based task that assesses students’ science reasoning skills. This virtual environment teaches the events related to volcanic eruptions and then scaffolds users through the process of collecting, interpreting, and using data to make predict the likelihood of a volcanic eruption. During the task, the user is presented with a simulated volcano, allowed to place seismometers (1-4 seismometers), and prompted to select a length of time to collect data (2 days, 2 weeks, 2 months). The data corresponding to their selections are presented and the user is asked to take notes and annotate that data by identifying important dates and the events that occur. After interacting with the simulation and taking notes, the user is engaged with a series of conversations that are adapted to the previous selections and created artifacts.
The adaptive conversations require the user to make comparisons, evaluate claims made by peers, explain logical connections between claims and evidence, and reason through the implications of hypothetical events. For example, the first conversation is a discussion about the amount of data included within one of the user generated notes and comparing it to an example note created by a peer agent (which has either more or less data than the user’s note). Within this conversation, the user must make a decision and justify which note they wish to use in the future for making a prediction. The second conversational pattern implements a sequence of judgment, justification, and claim generation (i.e., predict the likelihood of a volcanic eruption). Within this discussion the peer agent presents an erroneous claim with evidence and the user is asked to agree or disagree with the proposed claim and provide justification for that decision. Additionally, at the end of the conversation the user is asked to make their own claim and include valid reasoning with evidence from their notes. In the final set of conversations, the user is asked to provide a summary of their claim with explicit connections to the data along with the reasoning for selecting that particular data. After providing this summary, the user is posed with two hypothetical questions that ask about the ramifications of collecting more or less data and the impacts that would have on the accuracy of their claim.
The differences between these patterns and the evidence collected will be evaluated and compared, along with a discussion of the implications for conversation-based assessments.

Authors