Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Search Tips
What to do in Chicago
Personal Schedule
Sign In
X (Twitter)
Our goal is to develop a conversation-based assessment that integrates reading, writing and critical thinking skills essential for practicing argumentation. In this presentation, we will focus on the design of an open-ended conversation-based activity within the Seaball - Semester at Sea game prototype. This game is designed to collect a range of evidence about students' argumentation skills, which may have potential benefits for instructional practice. Methodological challenges to this approach will be examined for the purpose of improving future designs of conversation-based assessments.
Argumentation is a social activity of reasoning, a dialogue in which participants may take different positions and change their minds as it proceeds (van Eemeren, Grootendorst, & Henkemans, 1996). However, traditional classroom instruction in argument, which frequently emphasizes basic written composition and use of specific formal templates (Hillocks, 2002), does not capture the social aspect of argumentation that supports the development of sophisticated argument skills. As evidenced by a variety of empirical studies (e.g., Ferretti, MacArthur, & Dowdy, 2000; Nussbaum & Kardash, 2005; Persky, Daane, & Jin, 2003), students often make assertions with supporting reasons, but fail to include relevant evidence or consider alternative perspectives, and as a result they cannot critically evaluate people's arguments. Furthermore, traditional assessment of argument, which typically requires students to write an essay on a single prompt, provides little information about why students failed to accomplish this task, because only the final written product is evaluated.
To address this problem, we developed a prototype argumentation game designed for middle school students. Here, we describe an innovative approach to assessing argumentation skills, i.e., designing a conversation-based game activity which creates an authentic social context for argumentative discourse to occur, to provide a wider range of evidence of students' argumentation skills, including use of evidence and evaluation of others' arguments. Prior research indicates that computer-mediated conversations have the potential to improve student learning and motivation (e.g., Graesser, Person, & Harter, 2001). Although this argumentation game differs from a typical tutorial dialogue, it embeds scaffolding throughout the activity, which may have useful instructional implications.
A series of studies will be conducted with middle school students: (1) a path validation study (N=15) to identify and refine the dialogue models that guide the conversation paths; (2) a usability study (N=10) to identify problems in the activity directions or functionality, and to obtain initial feedback from students about their experience with the game; and (3) a pilot study (N=60) to compare and the conversation-based version of the game activity to a constrained, selected-response version, and to analyze how the patterns of student responses differ as a function of response format. Data collection and analysis will be completed in Fall 2014.
Our work presents an innovative approach to assessing and facilitating the development of argumentation skills, grounded in a social and dialogic framework. This work will not only yield important evidence about student performance in game-based argumentation tasks, but will also discuss lessons learned from the assessment design process.
Yi Song, Educational Testing Service
Jesse R. Sparks, Educational Testing Service
Wyman Brantley, Educational Testing Service
Colleen Foley
Margaret Vezzu, ETS
Tanner Jackson, Educational Testing Service