The interaction of multisensory cues on novel word learning

• In Event: Poster Session 05
  In Poster Session: PS 05 Topic - Language, Communication

Abstract

How might cues from multiple sensory modalities support early word learning? The current evidence is mixed. While multisensory cues facilitate perceptual pattern learning in infancy (Bahrick & Lickliter, 2000; Lew-Williams et al., 2019), they may negatively impact toddlers’ ability to learn words during book reading (Shinskey, 2020). This project explores the role of multisensory cues in word learning by measuring whether and how multiple sensory cues support known word recognition and novel word learning in a lab-based experimental task.
Toddlers between 24- to 30-months completed a novel word learning and known word recognition experiment consisting of three phases, (1) Exposure, (2) Teaching, and (3) Test. In the Exposure phase, toddlers initially viewed two novel objects for equal duration, one which they simply viewed (“See”), and another which was seen and touched (“See+Touch”). In the Teaching phase, participants’ eye movements were recorded as the previously exposed novel objects appeared on a computer-monitor and were labeled (a looking-while-listening (LWL) procedure). During Test, toddlers completed an eye-tracked word recognition task that assessed toddlers’ learning of the novel words, and recognition of known high and low sensory words. Novel test trials involved measuring gaze towards the two novel objects when one was labeled (Figure 1). Known word recognition trials either involved target objects that were highly-multisensory, by being associated with semantic features that activate multiple sensory modalities or low-sensory items, which were associated with semantic features that activate fewer sensory modalities. For example, according to semantic feature norms (McRae et al., 2004) a “High” modality item, POPCORN, includes gustatory, tactile, and visual semantic features, whereas a “Low” sensory modality item, BUTTON, includes only visual semantic features. Across all versions, the presentation and labeling of novel stimuli and all LWL trials were counterbalanced, yielding a fully balanced within-subjects design.
Preliminary analyses (n=31) indicated marginal differences in toddlers’ fixation towards the labeled novel objects in the See and See+Touch conditions, t(30) = 1.95, p = 0.061, d= 0.350, though the effect size estimate indicated a small advantage for the See+Touch condition. Accuracy proportions in the See+Touch novel word conditions slightly exceed chance, while Accuracy proportions in the See condition were below chance (Figure 1: “See+Touch” and “See”). This suggests that children, as a group, found the novel word learning task challenging. In the known word trials, recognition of high multisensory items was facilitated compared to low-sensory items, t(30) = 2.62, p < .001, d= 0.685, with both mean fixation proportions being above chance (Figure 1: “High” and “Low”), suggesting that multisensory cues play a role in word learning and/or retrieval over time. Finally, toddlers more accurately recognized familiar vs. novel words, t(30) = 4.58, p < .001, d= 0.822 (Figure 1). Further analyses are planned to explore if individual differences in vocabulary size and item knowledge influence word recognition and learning patterns.
In sum, the results provide preliminary evidence that multisensory cues can promote known word recognition within toddlerhood, though this multisensory “boost” may not be particularly robust at the earliest stages of word learning.

Authors

• Michelle Indarjit, Purdue University, West Lafayette
  Presenting Author

• Amanda Seidl, Purdue University
  Non-Presenting Author

• Arielle Borovsky, Purdue University, West Lafayette
  Non-Presenting Author