Trends in Student Mastery and Math Content Knowledge Through Spatial-Temporal Game Play: A 12-Year Look

• In Event: 43.015 - Game-Based Assessment: How Has the Field Matured Over the Past 10 Years?

Abstract

Research shows that learning with multiple representations fosters deeper learning within a domain (Ainsworth, 2006). Key to this robust learning is for students to make connections between these representations – conceptually understanding how representations relate to each other, and effortlessly making these connections (Koedinger et al., 2012). In this poster, we present a case study that longitudinally looks at game-based content and game design changes and their effects in a spatial-temporal educational game for math learning – ST Math. We leveraged student interaction data (logs) from ST Math to evaluate the changes and lessons learned in game design and implementation in schools over the course of 12 school years while looking at student usage patterns, game progress performance, and math test performance. We show practical measures that have been designed and implemented within the game – what has worked and what has not worked based on challenges in implementation and evidence of efficacy.

ST Math is a game-based instructional software in the preK-8 arena, using a spatial temporal (ST) approach for developing math comprehension and proficiency, using interactive visual manipulatives and animations of mathematical concepts. ST Math starts the student’s learning path with puzzles consisting purely of visual models, then progresses students along their learning path with language, symbols and complex tasks. ST Math has been used at an increasing number of schools since SY 2005-2006. In SY 2010-2011, a re-design of the platform (Generation 4) increased its content by 30%, standards coverage by more than 80%, and redesigned game responses from multiple answer choices to constructed responses. Using existing interaction datasets of students’ activities and gameplay features in ST Math (generic traces, game performance) from 2005 to 2017, we analyzed trends in student usage and student performance in math achievement tests (i.e., California Standard Test or CST, and Smarter Balanced Assessment Consortium or SBAC) of students whose math curricula included ST Math gameplay. Using time-series visualizations and analyses, results showed that prior to Generation 4, students have been found to achieve about 50% average completion progress in the game’s math learning objectives or curriculum.

Results showed that with the increased math content and standards coverage in Generation 4, teachers have increased usage inputs of logins and minutes for students. However, students were found to still have maintained around 50% completion progress. With the integration of ST Math in their curriculum over the years, results also showed that similar effect sizes on achievement tests have been observed for all student subgroups. In addition, teacher survey results about program use and satisfaction across the different game design changes were analyzed – the transitions observed between whole-class usage (where teachers can oversee) and class-rotation usage (where students are more on their own), and grade-level effect sizes. Such results from exploring longitudinal patterns of student game usage and performance in relation to their math content knowledge and mastery has provided insights for ST Math over the years in the design and re-design of not just game elements but also improvements in its implementation by teachers in classrooms.

Authors

• Andrew Coulson, MIND Research Institute

• Maria Ofelia San Pedro, ACT, Inc.