Digital Games as Laboratories: Modeling Student Development of Argumentation Skills Through Gameplay Data

• In Event: 65.077 - Game-Based Assessment and Learning of Argumentation Skills

Abstract

Objectives
Given their capacity to record players’ game-based actions, digital learning games can provide rich environments for investigating what players know and can do. Digital games have also garnered attention for their potential to support learning through use of situated contexts, and their ability to support transmission and adoption of epistemic norms – helping players to solve problems more like experts. The proposed presentation continues these lines of work by investigating a series of models of student cognition for evaluating evidence in the context of a digital argumentation game.

Framework
The investigation assumes a socio-cognitive view of learning (Greeno, 1998), and employs the notion of epistemic frames (Shaffer, 2006) while using evidenced-centered design (Mislevy, Almond, & Lukas, 2003) to generate and compare four models of player cognition.

Methods
The project investigated four models of student cognition for evaluating legal evidence in the context of iCivics’ digital argumentation game, Argument Wars. Through use of student game play data, conditional probabilities, correlation and Bayes nets, the project has been able to compare the four models for evaluating legal evidence.

Model 1 for example, describes how students incorporate criteria in their evaluation of evidence as they become increasingly sophisticated. Model 2 incorporates additional aspects of the evidence to include the presence of reasoning and the realism of the information. The model-comparison approach used here is analogous to work on diagnostic and explanatory modeling, e.g. Fischer (1973), Embretson (1994), and Wilson and De Boeck (2004).

Data Sources
Game play data was collected from (n = 250) students in grades 5-8 playing Argument Wars - a digital web-based game that pits players against an opponent arguing historic legal cases. The player presents a series of evidence cards to build their argument and win points from an automated judge. The cards can be categorized and scored in several ways. For example, in Model 1, the cards were categorized and scored according to three types of content: cards presenting 1) only unrelated information, 2) relevant facts, or 3) relevant legal precedent.

Results
During the project’s first trial, the support selected in each turn was recorded and categorized according to scoring rules dictated by each cognitive model. In Model 1 for example, cards presenting legal precedent were awarded highest scores; cards with only irrelevant information were awarded the lowest scores. In spite of the models’ substantive differences, early results show only small quantitative differences in their behavior. Correlations (Spearman’s rho) between values of the cards selected at turn 1, turn 2 and turn 3 and the players’ total score for example, reveal statistically insignificant differences between the two models.

Significance
The proposed presentation will contribute to discussions of the utility of digital learning games for research by making clear benefits they afford as a result of their potential for rich descriptions of student actions. With argumentation assuming a central position in the Common Core ELA and the Next Generation Science Standards, the presentation’s investigation of cognitive models for evaluation of evidence will be relevant to many as well.

Authors

• Seth Corrigan, GlassLab Games

• Alison Atwater, iCivics