Comparing the Impact of Different Digital Badge Designs on Learning From a School Geometry Game

• In Event: Learning With Static and Dynamic Visualizations

Abstract

Our goal was to investigate the impact of the design of digital badges on learning from a geometry game. Badges, i.e., markers of achievements (Li et al., 2012), can be viewed as one aspect of the emotional design of games (Plass & Kaplan, in press) that impacts motivational and cognitive outcomes. Based on a typology of badges we developed for this research, and grounded in Achievement Goal Theory (Linnenbrink & Pintrich, 2002), we conducted two experiments with random assignment of participants to treatment conditions in which we compared different badge types. Participants were students sampled from middle schools in ethnically diverse urban schools in the northeastern United States. Measures in both studies included the Situational Interest Survey (Linnenbrink-Garcia et al., 2010), the mastery achievement goal orientation subscale from the Patterns of Adaptive Learning Scales (PALS; Midgley et al., 2000), in-game performance, and pre- and post tests of geometry knowledge.
Study 1 examined the overall impact of badges on learning. Students (N=52) were randomly assigned to either play the badges or the no-badges version of Noobs vs. Leets (CREATE, 2014), a digital game to learn geometry, for approx. 25 minutes.
No-badges group performed significantly better on the posttest than the badges group, F (1, 21) = 6.28, p = .018, partial eta squared = .17. This effect was mitigated by a condition by mastery goal interaction, F (1, 31), = 5.49, p = .03, partial eta squared = .15: Mastery orientation was positively correlated with posttest score in the no-badges condition, with no relation in the badges condition. This suggests that badges can help students with low mastery goal orientation, but may hinder students with high mastery goal orientation.
Study 2 examined effects of interest and badge design on learning outcomes. Participants (N=85) were randomly assigned to play a performance, mastery or no badges version of the geometry game.
Students in the Performance Badges condition performed significantly better on the posttest than students in the Mastery Badges condition (p = .007); no-badges condition was not significantly different from either of the other conditions, F (2, 84) = 3.60, p = .032, partial eta squared = .089. There was also a significant interaction between Badges and Situational Interest, F (1, 84) = 4.50, p = .01, partial eta squared = .11. Learners with higher situational interest performed better with mastery badges; learners with low situation interest did worse with mastery badges. There was no relation between situational interest and learning outcomes in the other conditions.
Overall our results suggest caution in the use of badges in games for learning as they may negatively impact learning. Results also add to the nascent empirical research base by demonstrating the impact of different badge designs on learning, suggesting that performance related badges and mastery related badges show different learning patterns for different types of learners. Future research will need to be conducted to generalize our findings to other badge ecosystems, other learner populations, and other subject matters.

Authors

• Jan L. Plass, New York University

• Melissa L. Biles, New York University

• Bruce Douglas Homer, City University of New York