Engagement While Learning: The Multimodal Data Perspective on Interacting With a Virtual-Reality-Based Simulation

• In Event: Promoting Postsecondary Engagement, Persistence, and Participation

Abstract

Current research uses a multimodal data analysis approach to evaluate the emotional and
cognitive aspects of learning and highlight the impact of engagement on learning. In this study,
nursing students’ learning process with a virtual reality (VR) simulation was captured by facial
expression analysis, eye blink frequency, and self-reports. Combining these objective and
subjective measures to track students’ engagement during learning revealed: a) a significant
positive impact of interest level on post-test performance and b) a significant negative impact of
sadness and high attentional load (as measured by blink rate) on post-test performance; which
overall explained 57% of the variability of students’ post-test scores. Integration of multimodal
methodologies can inform instructional design.

Author

• Ilana Dubovi, Tel Aviv University