Assessing Skills Competencies Using Serious Educational Games

• In Event: Using Simulations to Prepare Teachers for Equitable Education

Abstract

The objective of this presentation is to discuss the design and methodological considerations underpinning automated assessment of skills competencies inside serious educational games (SEGs). SEGs have been shown to be effective in increasing both affect and cognitive achievement in learning settings when compared to other more traditional pedagogical practices (Lamb et al., 2018). Indeed, games have the potential to provide students with multiple opportunities for perspective on a given topic or domain, and can offer multiple, varied modalities for interaction with content material. However, such affordances necessarily introduce new challenges when it comes to assessment; namely, making sense of student choices and behaviors during gameplay. This emerging field of game-based performance assessment leverages computer technologies to detect and aggregate in-game measures as evidence for assessing cognitive skills and competencies (Bellotti et al., 2013; Michael & Chen, 2005). But to achieve this, data alone is not sufficient. Valid performance assessment of student skills and competencies requires thoughtful planning; an intersection of the dimensions of domain modeling, assessment design, game design, and model-construction for making inferences (Koenig et al., 2021; Koenig & Chung, 2016; Koenig, Lee, Iseli & Wainess, 2009). During this presentation, a serious educational game - called Teach SEG – will be presented, in which we highlight how each of the aforementioned dimensions were considered and integrated to develop a game to assess teacher competencies in classroom management. In this game, teachers (players) are placed inside various classroom contexts where autonomous (AI-driven) students engage in a corpus of behaviors to which the teacher must react and manage. The game, which plays out over the course of a single, 45-minute lesson (broken up into different phases), incorporates real-time assessment of player actions, with data parsed and fed into a Bayesian network where performance of player actions is scored and inferences of player competencies are estimated. The presentation will emphasize the significance of this work; namely, cross-disciplinary design considerations and practices that generalize to assessment of skills within any serious educational game design effort.

Author

• Alan Koenig, University of California - Los Angeles