Individual Differences in Infants Perception of the McGurk Effect

• In Event: Poster Session 01
  In Poster Session: PS 01 Topic - Perceptual, Sensory, Motor

Abstract

In the first year of life, infants’ perceptual and language development are intertwined. Previous research demonstrates that infants’ early sensitivity to audiovisual speech is associated with later receptive and expressive vocabulary (Bahrick et al., 2018). A striking demonstration of infants’ audiovisual speech integration is their ability to perceive illusory McGurk fusions (Burnham & Dodd, 1996; 2004). In the McGurk effect, an auditory /ba/ stimulus is temporally synced with a visual /ga/ stimulus, often resulting in a /da/ or /tha/ percept (McGurk & MacDonald, 1976). Although fusion effects have been evidenced in both infants and adults, some participants are non-fusers who demonstrate an auditory stimulus bias (i.e., /ba/).

Here, we examined individual differences in 33 (16 girls) 4.5-month-old’s perceptions of McGurk stimuli using a stimulus alternating habituation task (Singh et al., 2016). Using a within - subjects design, we were able to test infants’ preferences for two different contrasts, which to our knowledge has not been done for McGurk perception in infants. Infants were habituated to the McGurk effect (auditory /ba/, visual /ga/) and tested on two counterbalanced test blocks comprising two trials: one control trial (habituation stimulus) and one test trial (alternating between habituation stimulus and a congruent /da/ or /ba/). We hypothesized that McGurk fusers should dishabituate to the relatively novel /ba/ test trial while non-fusers should treat the auditory /ba/ as familiar and instead dishabituate to the /da/ test trial.

Planned single sample t-tests revealed that only difference scores for /da/ test trials were significant t(31)=3.253, p=0.003, suggesting that infants perceived an auditory /ba/ for McGurk stimuli, not a fused percept. Interestingly, we found individual differences in McGurk perceptions (Figure 1); 12 infants followed the trend of the group-level data (dishabituating to the /da/ test trial but not to the /ba/ test trial) suggesting an auditory /ba/ percept. The opposite pattern emerged for 11 infants, suggesting these infants perceived the canonical fused /da/. Additionally, 3 infants dishabituated to both /da/ and /ba/, potentially indicating alternate percepts (e.g., /tha/ or /bga/), and 7 infants lacked stable preferences in either block.

To investigate these observed individual differences, we constructed a dynamic neural field (DNF) model to simulate looking behavior for habituation and test trials in our task. We are currently tuning the model to identify parameter values that facilitate fusion percepts for some infants and non-fused auditory /ba/ for others. The model architecture is described in Figure 2. Using a DNF model, we can simulate the neural processes underlying audiovisual integration in speech perception, as well as working memory maintenance, and infants’ looking dynamics that mimic the task-demands in our behavioral experiment. Further, we are collecting vocabulary size measures at 13 and 18 months to correlate with the infants’ behavioral data. Finally, we are conducting a meta-analysis across the infant literature to accurately pinpoint the rates at which infants perceive fusion and non-fusion effects. Together, our behavioral data, computational model, and meta-analysis will better inform the role audiovisual speech processing plays in early language development.

Authors

• Ryan Cannistraci, University of Tennessee - Knoxville
  Presenting Author

• Aaron T. Buss, University of Tennessee - Knoxville
  Non-Presenting Author

• Jessica Sari Fleming Hay, University of Tennessee - Knoxville
  Non-Presenting Author