Designing a Game-Based Assessment Around Argumentation Learning Progressions: Seaball: Semester at Sea

• In Event: 65.077 - Game-Based Assessment and Learning of Argumentation Skills

Abstract

Objectives
Traditional assessment of argument, which typically requires students to write an essay on a single prompt, provides little information about why students failed to accomplish this task, because only the final written product is evaluated. Our goal is to explore an innovative approach to assessing students’ argumentation skills, which is intended to collect a wider range of evidence about students’ skills and to support instructional practice, as part of the Cognitively-Based Assessment of, for, and as Learning (CBAL) research initiative. We designed a game-based assessment (Seaball - Semester at Sea) that engages students in various argumentation tasks, such as recommending policies through the student council, and debating with game characters.

Theoretical Framework
To gather relevant evidence about students’ argumentation skills, we designed game activities around a set of argumentation learning progressions (LPs; Song, Deane, Graf, & van Rijn, 2013). Informed by cognitive and learning sciences research (e.g., Bereiter & Scardamalia, 1987; Graham & Perin, 2007; Hayes & Flower, 1980; Kuhn, 1991), argumentation LPs describe the qualitative shifts that occur as students reach higher levels of sophistication in four strands of skills: (1) Appeal building: understanding an audience’s values and beliefs; (2) Taking a position: developing a position and understanding other perspectives; (3) Reasons and evidence: using reasons and evidence to support an argument and to evaluate others’ arguments; and (4) Framing a case: organizing and presenting an argument logically. We used LPs as a general framework to determine the targeted skills and levels and sequences of the tasks.

Methods, Data Sources, and Results
In the Seaball game, players assume the role of students embarking on a journey aboard a research ship. As they travel, students will explore other countries, serve on the student council, and work with others to solve problems. This presentation focuses on the "Junk Food" scenario (Should we sell junk food onboard the ship?), which consists of five activities of varying difficulty. First, players interview peers onboard, and classify their opinions into ban or allow categories. Next, players select an expert speaker to address the students about junk food. In the third activity, players identify the main claim, reasons, and evidence in the speech. Players then make a recommendation to the student council (i.e., ban or allow junk food) with supporting reasons. Finally, players sort food items into junk food and healthy food categories, which involves using relevant evidence and evaluating people’s arguments.
We will conduct a pilot study with 60 middle school students on the “Junk Food” scenario in Fall 2014. The findings will provide evidence of the validity of argumentation LPs as tools for designing game-based assessments, and will inform revisions to the design of the game activities, to enhance measurement validity and student engagement. Data analyses will be completed by the end of 2014.

Significance
Our work presents an innovative approach to assessing and facilitating the development of argumentation skills. This work will not only yield important evidence about student performance in this game-based assessment, but also have useful instructional implications.

 

Authors

• Yi Song, Educational Testing Service

• Jesse R. Sparks, Educational Testing Service

• Wyman Brantley, Educational Testing Service