Where Badges Work Better: Findings From the Design Principles Documentation Project

• In Event: Democratizing Learning Through Digital Badges: Theoretical and Analytical Frameworks to Advance Design and Research

Abstract

Digital badges are transformative because they can contain specific claims and evidence and circulate in social networks (Casilli & Hickey, 2015). The objective of this presentation is to share the lessons learned from an extended study of the 29 winning badge system development proposals funded in the 2012 Badges for Lifelong Learning initiative that catalyzed much of the current interest in open digital badges.

This two-year ethnographic study was organized around using badges to recognize, assess, motivate, and study learning. The project documented intended practices (derived from content analysis of the proposals), enacted practices (from structured interviews with project teams once the badge development was underway), and formalized practices (from structured interviews and follow-up communication after the development period). Specific badging practices were clustered into more general principles and examined in light of other project outcomes.

The project yielded six general conclusions that will be illuminated in the presentation with examples from the various projects:

1.Badges work better in some places than others. Only a subset of the projects succeeded in creating the badge system outlined in the proposal; only some of those projects succeeded in creating the larger envisioned ecosystem envisioned (Figure 1).
2.Badges work better where content and technology already exist. The “responsive” projects that started with content and then built badges and technology were most successful, while the efforts that “layered” badges into existing educational content and learning technologies were generally successful (Figure 2).
3.Badges work better as informal credentials. Only one of the five projects that attempted to award formal credit for badges was able to formalize the practice. This was because they had already implemented program quality standards, attendance tracking, and portfolio assessment.
4.Badges work better when internally valued. Of the ten badge development efforts that proposed to seek formal external endorsements, only four were able to formalize the principle during the project period. The six projects that failed to gain external endorsements cited a range of obstacles.
5.Badges work better when they offer unique claims and evidence. Projects that developed badges whose claims and evidence were redundant with other information sources struggled to establish the value of those badges.
6.Badges work better where the learning is social and networked. The two most successful projects layered badges into existing content and technology that was inherently social in nature.

In summary, the most obvious practices for increasing the value of badges turned out to be the most difficult practices to enact and formalize. This is significant because badges are still not widely valued by colleges and employers. On the one hand, projects might redouble efforts to award formal credit and gain external validation. On the other hand, projects might pursue less formal paths for increasing the value of badges (see Davis & Singh, 2015). Given the growing interest in badges and their introduction into most learning management systems, this evidence will provide useful guidance for what our study showed to be very consequential decisions.

Authors

• Daniel T. Hickey, Indiana University

• James E. Willis

• Joshua Quick, Indiana University - Bloomington

• Christine Chow

• Katerina Schenke, University of California - Los Angeles

• Cathy Tran, MIND Research Institute

• Rebecca Itow, Indiana University