Combining Eye-tracking and Functional Near-Infrared Spectroscopy to Assess Children’s Word Recognition Skills: Lessons Learned and Future Directions.

• In Event: Poster Session 02
  In Poster Session: PS 02 Topic - Methods, History, Theory

Abstract

Word recognition is fundamental to children’s language development. To date, important technological advances in behavioral (e.g. eye-tracking) and neuroimaging (e.g. EEG/ERP, fNIRS) methods have allowed researchers to better study how word recognition develops throughout childhood. Here, we show how simultaneous multi-modal eye-tracking and functional Near Infrared Spectroscopy (fNIRS) neuroimaging can be leveraged to shed new light on the neurodevelopmental processes underlying early word recognition. We present findings from an innovative project that combined fNIRS (to investigate patterns of neural activation underlying word recognition in infants and toddlers) while simultaneously acquiring behavioral performance data using eye-tracking. By combining fNIRS with real-time eye-gaze data, we can match brain function to language abilities in children, allowing us to track the earliest indicators of language. Our study focuses on the methodological challenges and solutions, lessons learned, and recommendations for future steps in multidisciplinary, multi-modal techniques for child language research.
We have conducted several rounds of piloting and data collection using modified versions of a preferential looking paradigm to assess word recognition in children between 17 and 41 months old (data collection ongoing). A key design feature of our protocol is portability, which allows us to replicate a lab environment off-site in a familiar environment (i.e. participants’ home) and to recruit children who may not have easy access to our laboratory. During each session, children saw pictures of familiar objects (e.g., cookie, balloon) and heard speech stimuli of sentences instructing them to locate one of the objects in the pair (e.g., look at the cookie) while undergoing fNIRS neuroimaging (NIRx NIRSSport 2). Using a Tobii eye-tracker we measured children’s ability to accurately look at familiar objects on a screen. AnalyzIR toolbox (Santosa et al., 2018) was used to estimate the effect of audiovisual (linguistic) word recognition versus visual (non-linguistic) conditions in frontal and bilateral temporal language regions as well as correlations between eye fixation duration and neural activation patterns for word recognition with a linear mixed-effects model.
Behavioral data show that children’s accuracy during word recognition is above chance (M=0.60, SD=0.09 - similar to previous studies with the same age-range). Neural findings indicate that linguistic versus non-linguistic visual stimuli resulted in increased hemodynamic activity in the left temporal lobe -- areas that are critically involved in language processing. Specifically, we observed a significant increase of neural activation in the left superior temporal gyrus during audiovisual condition (picture+audio) relative to picture-only condition (b=75.135, p<.001).
As part of this project we have overcome a number of challenges including synchronization of eye-tracking and fNIRS devices and cross-training our research team in using both devices simultaneously. We will discuss the approaches that we found to be successful in addressing these areas in our presentation. Preliminary results to date demonstrate the feasibility of combining eye-tracking and fNIRS for studying the neurodevelopmental processes underlying word recognition in children and may provide critical information about how perceptual and linguistic networks organize during early childhood.

Authors

• Shakhlo Nematova, University of Delaware
  Presenting Author

• Katherine Richard, University of Delaware
  Non-Presenting Author

• Thierry Morlet, Nemours/Alfred I. duPont Hospital for Children
  Non-Presenting Author

• Giovanna Morini, University of Delaware
  Non-Presenting Author

• Kaja K Jasinska, University of Toronto
  Non-Presenting Author