Gameplay Sequence Similarity as an Indicator of Collaborative Problem-Solving Process in a Game-Based Learning Environment

• In Event: Game-Based Indicators of Learning Processes: Extraction Methods, Validity Evidence, and Applications

Abstract

Objectives
This study investigates middle school students’ collaborative problem-solving (CPS) process in a game-based science learning environment using an individual student’s gameplay action sequence.

Background
CPS involves not only cognitive but also socio-cognitive processes such as perspective taking, conversational grounding, behavioral mirroring, and joint action. Recent researchers have highlighted the collaboration process as a critical variable that influences whether groups can achieve their full potential for better CPS performance. Despite these benefits, little research has attempted to examine CPS processes captured within a computer-based learning environment which can offer opportunities to monitor collaborative processes and to provide real-time interventions to support learners in their knowledge construction process.

Methods
To compute a daily gameplay action similarity among students in a group, we used a similarity coefficient, Jaccard (Jac); that is, the Jac coefficients over the entire gameplay period were treated as each group’s action similarity trajectory. We employed a KmL cluster analysis to cluster all groups’ trajectory data to identify patterns of similar behavior traits. To further interrogate whether a cluster membership is associated with a group’s average knowledge gain, we conducted a chi-square test of association using two categorical variables: cluster membership and group knowledge gain (High, Low).

Data Sources
The sixth graders (N = 156) from a middle school played a serious game, Alien Rescue, over three weeks. In each group (a total of 70 groups), 2-3 students worked together to help six alien species displaced from different places in a distant galaxy by figuring out their new homes in our solar system. Before and after gameplay, an individual student’s comprehension of factual and applied scientific knowledge introduced in the game was measured using 24 multiple choice items. ​​The gameplay data were used to identify students’ navigation patterns as they engaged in the game.

Results
The clustering analyses suggest three distinctive group action similarity trajectories. The relationship between cluster and group knowledge gain was significant, which further illuminated how the groups’ CPS traits can be related to their learning performance. Each cluster’s characteristics shed light on deriving design considerations to promote students’ positive collaboration experience during CPS activities within serious games, and to engage teachers in facilitating students’ effective CPS process.

Significance
A unigram Jac coefficient quantifies a similarity between students’ gameplay actions in a group. We particularly hypothesized that a gameplay action similarity trajectory captures a group’s CPS process in this game. The results show it is a viable way in understanding different levels of group collaboration and can be further used as an indicator of students’ CPS process in game-based learning. This study also confirms KmL is one promising method to examine features at different time points generated from gameplay data, which all together provides insights into CPS process in serious games. The advantages and limitations of the methods employed in this study call for continued research on exploring potential analytical methods and scaling up the sample size to include a more diverse population to validate such exploratory CPS indicators.

Author

• Jina Kang, University of Illinois at Urbana-Champaign