Value-Added Approach to Game Research: Which Features Improve a Game's Effectiveness?

• In Event: 46.085 - Roundtable Session 15
  In Roundtable Session: 46.085-18 - Design Issues Regarding the Use of Games and Simulations for Learning and Assessment

Abstract

The objective of this meta-analysis review is to identify value added instructional features that improve the effectiveness of educational games as vehicles for promoting academic learning. The value-added approach to game research compares the learning outcome performance of students who learned by playing a game versus students who were assigned to play the same game with one instructional feature added.

Potential papers for this review of features to improve a game’s effectiveness came from searches of PsycINFO and ERIC using appropriate keywords such as “computer games,” “video games,” “serious games,” and “educational games,” the bibliographies of previous reviews of educational game research, and all “cited by” papers for classic educational game papers and existing metanalysis were also reviewed. From this pool, studies were selected for inclusion in the present analysis based on the following criteria: (1) The independent variable involves a comparison of a control group that is assigned to play a base version of a game versus a treatment group that is assigned to play the same game with one additional instructional feature added. (2) The dependent measure involves a measure of learning outcome performance such as answering questions or solving problems (rather than self-reports or in-game activity). (3) The paper reports the mean (M), standard deviation (SD), and sample size (n) for a measure of learning outcome performance for each group, or contains other statistical information that allows for computing a value of effect size (e.g., d or r). (4) The instructional content is in an academic area.

This review of value-added game research identified five promising features that tend to improve learning from computer games: modality in which words are spoken rather than printed (d = 1.41), personalization in which words are in conversational style rather than formal style (d = 1.54), pretraining in which key concepts are defined in advance (d = 0.75), coaching in which online help is provided during game play (d = 0.68), and self-explanation in which students are prompted to reflect during game play (d = 0.81). Two features were unpromising: immersion in which virtual reality features are added (d = −0.14) and redundancy in which printed text is added to the screen to correspond to narration (d = −0.23). Those negative effect sizes indicates that the control group performed better than the treatment group. Five features have not yet been shown to be promising based on one to four studies: competition which involves adding scores and prizes, segmenting in which the game progresses in difficulty, image in which pedagogical agents appear on the screen, choice in which learners determine game features, and narrative theme in which the game is presented within a rich storyline. Overall, the value-added approach and meta-analysis are useful methodologies for identifying effective features of value added computer games for learning.

This research has been reviewed and approved by my university review board (IRB).

Author

• Richard E. Mayer, University of California - Santa Barbara