Audio Effect on Young Learners’ Multimodal Reading Comprehension and Processing: L1-L2 Eye-tracking Comparison Study

• In Event: AERA Roundtable Session Friday 1:45 pm - Gold 2
  In Roundtable Session: Language Pedagogies Across the Content Areas

Abstract

Multimodal input can improve reading engagement but also imposes greater cognitive demands on young learners. Prior research yields mixed effects of audio and visual scaffolds, while often overlooking the influence of image-text relationships and differences in L1 and L2 context. This eye-tracking study with 31 children (ages 9-12) compared comprehension and fixation duration across reading-only and reading-while-reading conditions in L1 Chinese and L2 English. Results show that audio significantly enhances comprehension, especially in L1. Eye-tracking data further reveal that, with audio present, L1 readers prioritize complementary images, whereas L2 readers fixate longer on concurrent, redundant images. These findings underscore the pedagogical value of audio support and the importance of considering image-text relations when designing multimodal materials for young learners.

Authors

• Yiwen Sun, Hong Kong Polytechnic University

• Nuo Li, The Hong Kong Polytechnic University

• Xinhua Zhu, Hong Kong Polytechnic University

• Yi Guan, Hong Kong Polytechnic University