Poster #11 - Word Comprehension: Initial Contribution of Salience and Word Cues

• In Event: 3-159 - Poster Session 12
  In Poster Session: PS 12 Section - Attention, Learning, Memory

Integrative Statement

A large literature using a wide variety of different tasks and contexts, across infants from 3 to 13 months, shows that heard names cause infants to look at visual objects. These result from looking behaviors that robustly indicate infants’ initial receptive knowledge of some word-object correspondences (Bergelson & Swingley, 2012; 2013; Tincoff & Jusczyk, 2012; Ferry, Hespos, & Waxman, 2010). Findings from this literature demonstrate that attention to referents when hearing words is a critical skill for learning words and becomes more efficient with time and experience (e.g., Bergelson & Swingley, 2012; Weisleder & Fernald, 2013). Other research suggests that infants’ attention is primarily driven by exogenous factors (bottom-up/saliency driven) as top-down control mechanisms in infants are relatively limited (e.g., Amso, Haas, & Markant, 2014; Colombo, 2001). This demonstrates that orienting to referents when hearing words involves developmental departure from bottom-up attention (Hirsh-Pasek, et al., 2000; Samuelson & Smith, 2000; Smith, 2000). However, we know little about the developmental origins of the “heard name to visual object” attention. The present study aimed to document the developmental trajectories of the shift: from bottom up attention driven by visual saliency to top-down attention driven by the link between heard words and seen objects.

In the present study, 74 infants age ranging from 3 to 19 months participated in a “looking while listening” task (Bergelson & Swingley, 2012; 2013; Fernald, Zangl, Portillo, & Marchman, 2008; Swingley, 2011). Infants viewed two discrete objects, displayed through a computer screen with eye-tracking, where object saliency and labeling (“Do you see the apple?”) were varied to examine whether or not infants look toward the named item. The two displayed objects included photographs of familiar toys (infants had previous exposure) and photographs of novel instances of the same category.

We computed saliency maps and heatmaps based on the multi-dimensional measure of dense contrast (salience) taken from task images (Itti, Koch & Niebur, 1998) and attention density taken from infant fixations and duration of attention (Dalmaijer, Mathôt, & Van der Stigchel, 2013), respectively. We measured observation intensity and latency to look at the correct referent after labeling as a measure of receptive word knowledge (top-down attention) and computed a similarity measure between the saliency and heatmaps to estimate bottom-up attention. The key question is how the saliency and heatmap align, and whether that changes with age. The results suggest the hypothesized shift— infants dependent on dense contrasts (saliency) initially, and gradually shift to top-down attention. The results also point to the role of pre-exposure to objects as the trend suggests top-down (linguistically driven) attention in trials with the depiction of familiar toys. This indicates the potential role of visual familiarity in shifting to top-down attention. The results provide new evidence on how early bottom-up attention and object exposure contribute to early performance during a looking while listening task, and will be discussed in the relation to the implication for other attentional measures.

Authors

• Aakash Patel, University of Houston
  Presenting Author

• Sunbok Lee, University of Houston
  Non-Presenting Author

• Hanako Yoshida, University of Houston
  Non-Presenting Author