Design Principles and Relevant Resources for Recognizing, Assessing, Motivating, and Studying Learning With Digital Badges

• In Event: Innovating Education Practice Through Digital Badges: Recent Research, Current Practices, and Future Directions

Abstract

Purpose
This paper presents the design principles for using digital badges that emerged across the 30 projects that were funded to develop digital badges in the 2012 Badges for Lifelong Learning Initiative. The paper also presents a review of the relevant research literature for the various principles.

Theoretical Framework
The 30 badge design projects had to refine their plans to accommodate things they had not anticipated and uncovered unimagined (and unimaginable) new opportunities. Research on design rationale (e.g., Kruchten, 1995) has shown that most of the knowledge generated when designing complex systems “evaporates” as features evolve and teams dissolve. This project captured this knowledge so it could be synthesized and shared.

Like prior studies (Kali, 2006), this project distinguished between specific practices within projects and more general principles across projects. Like contemporary design-based research methods, this research connected practices to relevant features of the project contexts. Following Maxwell (2006), this contextualized relevant research literature to specific practices and principles.

Methods and Data Sources
The project first defined a manageable set of badge design principles to work with. Deliberation resulted in a focus only on badge functions that concerned learning, and then on four categories of functions: recognizing learning, assessing learning, motivating learning, and studying learning. The thirty submitted proposals were analyzed to identify intended practices in each area. Interviews with project staff uncovered the enacted practices and relevant features of project contexts. The various practices were then clustered into 6-10 more general design principles. Relevant research resources were then identified for each principle.

Results
An interactive database was organized around the 6-11 principles in each area, with links to specific project practices, relevant project features, and relevant research literature. Rather than “best practices” this research identified enacted practices that appear to be appropriate for the particular context, and more general principles in which to cluster and discuss those practices.

Additionally, this study revealed important constraints in each area as follows. Recognition practices are constrained by existing ecosystems; assessment practices are constrained by recognition practices; motivational impact is constrained by recognition and assessment, but motivational consequences are defined by assumptions about learning; research methods are constrained by recognition, assessment, and motivational practices, but evidence is defined by assumptions about learning, and badges contain actual evidence of learning that can be used immediately to study and evaluate that learning.

Significance
While interest in badges has exploded, there is little systematic guidance; debates have focused on the enduring debates over incentive. This project will provide useful knowledge and help create a knowledge community for using digital badges in ways that are productive and avoiding strategies that are unproductive.

More generally, this research provides a model of practice for extracting and sharing the tacit design principles that emerge from cutting edge educational technologies. It provides a model for moving beyond the faddishness and general debates with such innovations and producing truly useful knowledge.

Authors

• Daniel T. Hickey, Indiana University

• Rebecca Itow, Indiana University

• Andrea M. Rehak, Indiana University

• Katerina Schenke, University of California - Irvine

• Cathy Tran, University of California - Irvine