Designing Game Activities to Assess Students’ Argumentation Skills

• In Event: Designing Game-Based Assessment Around Learning Progressions

Abstract

A game-based assessment of students’ argumentation skills, situated in the Cognitively Based Assessment of, for, and as Learning (CBAL) framework, was developed. The game’s theme is a Semester at Sea program, during which students explore other countries, learn about different cultures, and work in teams to solve problems. Game activities require players to demonstrate their argumentation skills. Throughout the “voyage,” players will engage in debates with game characters, serve on the ship's student council, and complete various tasks and scenarios involving argumentation.
The goals of this game project are to create engaging argument-related tasks, and to gather relevant evidence about students' argumentation knowledge and skills. To achieve these goals, the game was designed around argumentation learning progressions. The progressions function as a framework for assessing student competencies and an evidence-centered design model for assessment development (Deane, 2011; Mislevy, Steinberg, & Almond, 2003).

Based on cognitive and learning sciences research in the domain (e.g., Bereiter & Scardamalia, 1987; Graham & Perin, 2007; Hayes & Flower, 1980; Kuhn, 1991), argumentation learning progressions describe the qualitative shifts that occur as students reach higher levels of sophistication in four strands of skills:
• Appeal Building: understanding an audience’s interests, values, and beliefs
• Taking a Position: developing a position and understanding other perspectives
• Reasons and Evidence: using reasons, evidence, and examples to support an argument and to evaluate others’ arguments
• Framing a Case: organizing and presenting an argument logically

The game will include small challenges in which players can exercise one of the specific skills, and larger scenario-based tasks that require players to coordinate multiple skills. The argumentation learning progressions play a critical role in game design because they help determine the targeted skills in the assessment and match innovative items to particular levels of performance. For example, at the lowest level of the progressions, students are expected to select effective appeals for an intended audience. To assess students’ proficiency with this skill, we have developed a "persuading parents" activity. Specifically, players will help game characters identify the most convincing arguments to address their parents' concerns about the trip (e.g., concerns about safety, academic learning, or nutrition). When players successfully persuade a character’s parent by selecting the most convincing argument, the character is given permission to join the trip.

Furthermore, the learning progressions help us establish appropriate sequences of the tasks, and decide when to provide scaffolding tools in the game. For instance, an advanced skill in the learning progressions is critiquing an argument, which empirical studies suggest develops fairly late (e.g., Knudson, 1992; McCann, 1989; Nussbaum & Kardash, 2005). This implies that we either incorporate scaffolding tools to support students in performing critiquing tasks or introduce critiques only at a higher (i.e., intermediate) level in the game.

Data from usability studies will provide evidence about the validity of the tasks, quality of game features, and information about student performance levels aligned to the argumentation learning progressions. Analyses and results will appear in the final paper, with implications for instruction and assessment.

Authors

• Yi Song, Educational Testing Service

• Jesse R. Sparks, Educational Testing Service