Multimodal Data Fusion to Detect Students' Cognitive-Affective States During Educational Gameplay

• In Event: AERA Poster Session 6
  In Poster Session: Games for 21st Century Learning (San Diego)

Abstract

We implemented multimodal data fusion to build a prediction model that detects students’ cognitive-affective states (i.e., engagement and stress) during educational gameplay. Cognitive-affective states mediates students’ learning in gameplay. Researchers have sought to predict students’ dynamic cognitive-affective states by using multiple data sources. However, limited research explored how to systematically collect, synthesize, and interpret multimodal data in game-based learning research. In this study, we implemented multimodal data fusion to detect students’ cognitive-affective states during educational gameplay. This paper will present how multimodal data fusion improves the performance of the prediction models in assessing cognitive-affective states.

Authors

• Jewoong Moon, The University of Alabama

• Fengfeng Ke, Florida State University

• Zlatko Sokolikj, Florida State University

• Ibrahim Dahlstrom-Hakki, TERC