Interest Analytics and Metrics for Measurement in Games

• In Event: Game-Based Motivational Analytics and Metrics

Abstract

Objectives. The paper will present a theoretical framework on interest and empirical results of how to assess interest using both game analytics and metrics.

Theoretical Framework. Interest is conceptualized by either stable/trait-like (e.g., career interests) or fluid/dynamic/state-like (e.g., interest in games). Trait career interest has been linked with success in the world of work (Volodina & Nagy, 2016). In contrast in game-based contexts, interest is viewed as mainly a product of game characteristics.

Method. Most promising for trait-like situational-based interest measures for young adults is either the Job and Occupational Interest in the Navy (JOIN; Farmer et al., 2006) or the PISA (OECD, 2012) to measure career and content interests, respectively. We used JOIN measure, as our research is focused on high school students who may have an interest in the Navy. For state interest, measurement is based on AUTHOR’s work (see Figure 1). For example, evaluating player choice (Schwartz & Arena, 2013), mimics measures of career interest (e.g., Dawis, 2001) where players are asked to choose among many career interests.

We have implemented a subset of these analytics and metrics in a game called the Navy Life Game. The game operates with two tiers: Tier 1 is a fun and engaging fantasy adventure game and Tier 2 is a Navy job familiarization game. The second tier of the game provides opportunities for players to earn currency (e.g., special powers) that can be used to be successful in the Tier 1. With respect to user testing, we briefed a Navy admiral on our approach, which we conceptualized as a form of user testing with an organizational stakeholder. We also tested high school students (the intended audience for the game) in a small-scale usability study to gather information on the usability and playability of the Tier 2 game.

Results. The results with the organizational stakeholder indicated strong support for our technical approach and increased funding. The results with high school students indicated that they were somewhat or not very interested in (a) the Navy in general, (b) Navy as a topic, or (c) the Navy as an occupation. Participants reported being interested or very interested in games and were experienced in playing games.

Scientific or Scholarly Significance. The analysis indicates that our general approach to the measurement of interest as an outcome variable has merit. Our interest metrics for games could be used by other researchers and help to advance the state of the art in this area. The two-tier approach to game design is also promising as indicated by user testing.

Authors

• Harold F. O'Neil, University of Southern California

• Katerina Schenke, University of California - Los Angeles

• Alan Koenig, University of California - Los Angeles

• Stephen Watson, U.S. Navy