An investigation into the effects of conflicting emotional cues on toddlers’ visual emotion perception

• In Event: Poster Session 07
  In Poster Session: PS 07 Topic - Social, Emotional, Personality

Abstract

Emotion understanding is a crucial skill for social competency. The communication of emotion is dynamic and occurs across multiple channels such as face cues, tone of voice, a speaker's choice of words, and body language. When these cues are in conflict, interpreting emotion can become challenging. Emotion understanding in response to this kind of conflict has been investigated unimodally (e.g., visual domain – with facial expression and body posture, auditory domain – with prosody and tone), and multimodally (e.g., with competing visual and auditory cues). Less is known, however, about the development of emotion understanding when cues are in conflict. The current study examines toddlers’ visual emotion understanding when the semantic emotional meaning of a sentence is incongruent with the tone of voice in which it is spoken.
Thirty-three monolingual English-speaking children between 22-26 months (15 female, Mage = 23.8 months) were presented with 12 10-s trials consisting of side-by-side images of a person's face on a computer screen while an audio clip was played and eye movements were recorded. Audio clips consisted of sentences carrying specific emotional content was spoken in a tone incongruent with that meaning. For example, “My ice cream fell on the ground, it’s all ruined” carries sad emotional semantic content but was spoken in a happy or angry tone of voice. Images remained on the screen for the entire 10 seconds. One image displayed a person’s facial expression matching the tone of the audio, and the other image displayed that same person’s face making an expression matching the semantic content of the sentence.
We hypothesize that participants will show an overall looking preference for the face which matches the tone of voice vs. the face which matches the semantic content and will test for this using paired-samples t-tests. We further hypothesize an effect of age, due to increased experience with and sensitivity to language, which will be probed using regressions to assess whether, as age increases, participants show increased preference for semantic matches. As it is also possible boredom and familiarity with the array may influence looking behavior, we will follow up with exploratory analyses examining the effects of child attention and visual exploration across the full trial by comparing numbers of fixations across the two halves of the trial. With data collection complete, Fall 2020 will be dedicated to processing data and Winter 2020 to analyzing them, all of which will be complete by the April 2021 meeting.
Considering the myriad of emotional information presented in daily life, it is important to investigate how children make sense of the world around them, even when input may be challenging to interpret. This study is one of the first to investigate young children’s visual exploration when presented with contradicting emotional verbal cues. Results will inform our understanding of the development of complex emotion understanding and will provide insight into what cues young children find more salient—what is said vs. how it’s said.

Authors

• Shannon M Brady, University of California - Riverside
  Presenting Author

• Marissa Ogren, Rutgers University - Newark
  Non-Presenting Author

• Scott P Johnson, UCLA
  Non-Presenting Author