The Effect of Feedback on Learners' Behavior and Performance in a Digital Game

• In Event: 61.015 - Embracing and Modeling the Complex Dynamics of Motivation and Engagement: Contextual, Temporal, Dynamic, and Systematic

Abstract

Theoretical Framework & Purpose
The feedback students receive during learning has important implications for their subsequent performance and on their motivation (Hattie & Timperely, 2007; Shute, 2008). The provision of feedback does not occur in a vacuum; instead, it is the result of a learner’s previous experience. The Knowledge-Learning-Instruction framework (KLI; Koedinger, Corbett & Perfetti, 2012) describes the relation between instructional events (observable characteristic of the learning environment), learning events (unobservable event within the learner) and assessment events (observable characteristic of the assessment environment). Within this model, feedback is the result of both an assessment event and of previous learning events. We expand upon the KLI framework by incorporate motivational outcomes of learners specifically as it relates to their engagement in the game(defined here by how much students liked it, how much they want to continue playing it, how much effort they put forth, and perception of difficulty).
The purpose of this study is to evaluate the effectiveness of specific types of feedback on learner’s engagement and achievement in a physics game. This proposal takes advantage of event-level game-play data to understand the effectiveness of particular types of feedback on learner’s in-game behaviors and their eventual achievement and engagement. Additionally, we seek to understand the moments during gameplay that represent the best time points at which to give feedback.
Method & Data Analysis
Data from this study come from children’s (ages 5-7) gameplay in a PBS Kids game called Fish Force (see Figure 1). Participants were recruited from Boys and Girls Clubs serving low-income communities in Southern California. Data collection is currently ongoing and will be completed this month (anticipated sample size of 80). Participants will play Fish Force for 20 minutes a day, 5 days a week, for 1 week. Daily assessments of engagement (4-items) are administered.
Event-level data (roughly 2,000 events per child per sitting) allow us to understand what events preceded the feedback and what events followed the feedback. Children play on average 18 levels per sitting. There are 80 different feedback phrases given to players ranging from general (“can you get the plushie onto the target?”) to specific (“what if you used less force?”).
We will first evaluate the association between feedback type and subsequent performance by observing the fixed effects of specific types of feedback on achievement in a multilevel modeling framework where events are clustered within individual. Next, to incorporate the time-series nature of the data, we will incorporate the autoregressive paths between events (feedback and achievement). Finally, we will use Dynamic Structural Equation Modeling (DSEM, Asparouhov, Hamacker, & Muthen, 2018) to investigate the association between type of feedback and player’s subsequent behavior while incorporating information about the player’s previous behavior. Figure 2 shows a snapshot of the model as well as the coefficients that are estimated. Specifically, we focus on the parameters that relate to the within-person cross-lagged associations ΦAF and ΦFA to understand the association between feedback and subsequent achievement and the between-person covariances between feedback, engagement, and achievement (ΦAE and ΦFE).

Authors

• Katerina Schenke, University of California - Los Angeles

• Tianying Feng, University of California - Los Angeles

• Gregory K.W.K. Chung, University of California - Los Angeles

• Jeremy Roberts, PBS KIDS Digital