Tracing the Design and Testing of a Game-Based Learning Environment for Upper Elementary Students

• In Event: Advances in Assessing Game-Based Learning

Abstract

Purposes of the Work

The aim of this paper is to trace the iterative design and testing of an intelligent game-based learning environment (GBLE) that promotes problem solving and science learning in elementary students (5th grade). While evidence to support the positive affective impact of educational games has been mounting (e.g., Barab & Dede, 2007; Hickey, Ingram-Goble & Jameson, 2009; Ketelhut et al. 2008), little is known about the cognitive impact of educational games. In this paper we describe the constraints and affordances of our GBLE as we begin to unravel the behavior and cognition of its participants.

Our Perspectives

With the ultimate goal of implementing an empirically-based research program to study elementary students’ problem-solving processes and engagement with STEM content, our efforts are guided by the attributes of design-based research (e.g. Brown, 1992; Cobb, et al., 2003). We take a multi-level approach to the assessment of iterative refinements of our GBLE.

Methods

CRYSTAL ISLAND: UNCHARTED DISCOVERY is a NSF-funded action-adventure learning environment that blends problem solving, exploration, role-playing, and strategic action. During gameplay, students undertake quests that situate learning and advance the storyline. Currently, the learning environment’s curriculum focuses on landforms and map skills.

(a) (b)
Figure 1. Representative screenshots of (a) the environment (b) the user’s map.


Data has been garnered from focus groups, pilot-testing here and abroad, and two multi-school studies (N= 617). Our studies have employed a suite of instruments including: a Content Knowledge test, in-game Problem-Solving measures (e.g., goals accomplished, number of goal-related actions, and navigational efficiency within the CRYSTAL ISLAND), and a Self-Efficacy scale. Additionally, we use student trace data to assess student learning from in-game decision records.

Results
Preliminary results from our last round of testing indicate significant learning gains, which varied by condition (no scaffolding, engagement scaffolding, problem solving scaffolding, both types of scaffolding) as measured by the content knowledge test, map usage test, and problem solving-test based on Pólya’s (1945) model. While ANCOVA analyses demonstrated no differences between conditions for the map (F(3, 329) = .29, p > .05) and content tests (F(3, 329) = .21, p > .05), there were differences between conditions for the problem-solving test (F(3, 329) = 2.79, p < .05). Interestingly, according to simple contrasts, which compared the experimental conditions to the control, for the problem-solving test only the engagement condition differed significantly, demonstrating higher learning gains (t(162) = -2.84, p < .05, r = .22). The implications of these findings and ideas for the next iteration of the game will be discussed if the symposium is accepted.

Significance of the Work

Our work answers this year’s conference call for innovative research used to improve education and serve the public good; improving the quality of students’ engagement with STEM content has become a national priority. Methodologically, our adherence to the tenets of design-based research helps elucidate key principles for the effective design and assessment of GBLEs, including traditional measures and computer trace data.

Authors

• James Lester, North Carolina State University

• James Minogue, North Carolina State University

• John L. Nietfeld, North Carolina State University

• Hiller A. Spires, North Carolina State University