Using Game Play to Measure Children's Social and Emotional Learning: A Validity Study

• In Event: Designing and Validating Games to Measure Social and Emotional Learning: Validation Results and Lessons Learned

Abstract

Objective
This study explored the extent to which three games, designed specifically for measurement, could measure SEL skills. We asked: To what extent do game-based indicators reflect children’s ability to engage in SEL practices?

Background
Many states and districts now mandate yearly SEL assessments (Hull, 2013) and thus there is a growing need to develop SEL assessments, especially for young children. Digital games have the potential to serve as unobtrusive, easily adopted, collectible measures of SEL. Well-designed games can maintain children’s engagement, elicit effortful performance, and provide unique opportunities to demonstrate SEL.

Methods
Our sample included 144 four- and five-year old children (and their parents). We used a single-group design wherein participants completed external SEL measures (parents completed indirect measures and children a direct measure) and played the SEL games (children only). Each game was designed to measure different aspects of SEL. Tower of Cows (TOC) targeted persistence. Muffy’s Party Planner (MPP) targeted perspective taking. Kindergarten Helpers (KH) targeted understanding differences among people.

Data Sources
We used three measures of SEL: the Affect Knowledge Task, Shortened (AKT-S) (Denham et al., 2020), a direct assessment measuring emotion labeling and perspective taking; the Social Skills Improvement System: Social-Emotional Learning Edition (SSIS-SEL) (Elliott et al., 2017a, 2017b), an indirect assessment which included a social awareness subscale; and the Preschool Learning Behaviors Scale (PLBS) (McDermott et al., 2000, 2002), an indirect assessment measuring attention/persistence, competence motivation, and attitude towards learning.

To measure in-game performance in each game, we created game-based indicators (GBIs), variables extracted from the game telemetry (clickstream data). Table 1 presents the GBIs and their descriptions.

Results
We used correlations and item response models to examine students’ responses in the gameplay. Results are organized by game.

For TOC, no relation was found between any of the GBIs and the external subscale of persistence/attention of the PLBS. However, the attitude toward learning subscale was positively related to the number of correct first attempts and negatively related to the number of misses in TOC.

For MPP, positive, significant, and small (rs ~ .2) correlations were found between GBIs and two AKT-S subscales. No relations were found between GBIs and the social awareness subscale of the SSIS-SEL. Players’ propensity for identifying positive emotional responses was also related to higher scores on the AKT-S.

For KH, no relation was found between the GBIs and the subscale of social awareness of the SSIS-SEL.

Significance
The general results of the relations between game-based indicators and associated external measures for each game were mixed. The game with the most promise for measurement was MPP, as it was the only game with significant relationships found between the game-based indicators and the external measures. These results suggest that children who are more emotionally mature (as measured by the external SEL measures) also perform higher in MPP.
 

Authors

• Elizabeth Redman, University of California - Los Angeles
  Presenting Author

• Charles Parks, University of California - Los Angeles
  Non-Presenting Author

• Joanne Michiuye, University of California - Los Angeles
  Non-Presenting Author

• Yon Soo Suh, NWEA
  Non-Presenting Author

• Gregory K.W.K. Chung, University of California - Los Angeles
  Non-Presenting Author

• Junok Kim, University of California - Los Angeles
  Non-Presenting Author

• Noelle Griffin, University of California - Los Angeles
  Non-Presenting Author