Iteration of Evidence-Centered Game Design for 3-D Geometry Game Environment

• In Event: 43.015 - Game-Based Assessment: How Has the Field Matured Over the Past 10 Years?

Abstract

Traditional assessment is often demotivating or disengaging. The community has called for action to develop alternative ways to assess where students can have engaging and authentic experiences with the assessment process and feel less anxious about assessment, while not losing the rigor of inferences that we make (Shute & Becker, 2010). The field of game-based assessment (GBA) has emerged to explore the benefits of playful methods of assessing learning using games as a vehicle (Shute, Ventura, Bauer, & Zapata-Rivera, 2009). In spite of these research and development efforts, classroom assessment at schools hasn’t changed much and the use of GBA in schools is very limited (Takeuchi & Vaala, 2014). Furthermore, because the formal evidence-centered design (ECD) approach often starts with understanding the constructs in abstract of actual data, it can be at odds with the approaches that are emerging in educational data mining (EDM) and learning analytics (LA), where learners’ actions can be classified using machine learning algorithms in relation to target constructs. We aim to address two broad issues: How can we develop GBA to be useful and usable by students and teachers in schools? And How can we iterate the existing GBA development frameworks, so that we can leverage the existing knowledge of LA and EDM?

Our project develops a GBA platform that can be implemented across the 9th grade math curriculum as frequent and ongoing formative assessment. Ultimately, our aim is that this platform can be used in place of exit slips--a formative assessment technique commonly used by teachers, by providing quick formative assessment (Conderman,& Hedin, 2012). Our assessment aims to explicitly measure common core Geometry standards (e.g. visualize relationships between 2D and 3D objects) and relevant reasoning skills (e.g. spatial reasoning), while simultaneously measuring valuable attributes and behaviors in the game that are connected to lifelong outcomes such as persistence and creativity.

Another intellectual contribution of our project to the field is how we are iterating the evidence-centered game design (ECgD) framework (Mislevy et al., 2012) by incorporating frequent playtesting and the use of learning analytics, while rapidly developing formal assessment models through this iterative process. While ECgD is a useful framework that allows game and assessment designers and system engineers to establish coherence across all aspects of development, much of the effort of formalizing assessment models (e.g. competency model) can make the process less iterative and more linear. Additionally, this process often delays psychometric evaluation of the assessment machinery until a time when the game is fully developed. By contrast, we are applying an iterative process with a rapid prototyping methodology with multiple playtests, where for each one of them, we conduct a close analysis to ensure we are maintaining the balance between game design, data modeling, and assessment algorithms. Lastly, by incorporating LA and EDM approaches, we are intentionally designing interactions and interfaces in the game that can be used to connect with validated detectors for both off-task and engagement behaviors (Wixon, Baker, Gobert, Ocumpaugh, & Bachmann, 2012; Paquette & Baker, 2017).

Authors

• Yoon Jeon Kim, Massachusetts Institute of Technology

• Jose A Ruiperez-Valiente

• Louisa Rosenheck, Massachusetts Institute of Technology

• Eric D. Klopfer, Massachusetts Institute of Technology