Game-Based Measures of Children's Learning Process and Performance in a Physics Game: Design, Utility, and Validity Evidence

• In Event: Game-Based Measures of Learning Processes: Challenges, Methods, and Solutions

Abstract

Although gameplay can improve young children’s understanding of physics and engineering concepts (Grindal et al., 2019), less clear is whether gameplay can be used for measurement purposes. Thus, we examined the extent to which GBM could be used as indicators of children’s learning and performance. 115 children (4-6 years old) played a physics game and completed an external knowledge test and an external performance task, both covering height and friction concepts.

The game presented two slides, one controlled by the player and one by the game. The goal of each level was to either beat, lose to, or tie the game characters as they slid down the slide, by adjusting the slide’s height, surface friction, or both. Three classes of measures were developed: (a) usage measures of interest to game producers, as specified by game stakeholders; (b) “common measures” representing game performance and game progress (Chung et al., 2015, 2018); and (c) game-specific measures representing the quality of the player’s slide adjustments.

Common GBMs that related to the performance task were number of levels completed (r = .24, p < .05), mean time spent on a level (r = -.32, p < .01), and total number of correct first attempts (r = .32, p < .01), consistent with our prior work. Game-specific measures included whether a player’s adjustment was productive (getting closer to the solution) relative to the ideal solution or unproductive (getting further from the solution) and related to performance on the knowledge test (r = .26, p < .001). Furthermore, when learners were grouped into high and low performers based on the game-specific measure, these groups differed significantly in performance on the knowledge test (d = 0.71, p = .002) and performance task (d = 1.00, p < .001). These results suggest that game-based measures can be useful for detecting differences in knowledge.

Authors

• Elizabeth Redman, UCLA/CRESST

• Gregory K.W.K. Chung, University of California - Los Angeles

• Tianying Feng, National Center for Research on Evaluation, Standards, and Student Testing

• Charlie Parks, University of California - Los Angeles

• Kilchan Choi, University of California - Los Angeles