Poster #35 - Sign-naïve and native signing infants’ visual preferences for signing avatars

• In Event: Poster Session 11

Abstract

Background: Virtual signing avatars have the potential to provide language input to deaf infants, many of whom are born to hearing parents who do not know sign language. Signing avatars vary widely in their appearance, naturalness of their movements, and facial expressions—all of which may potentially impact infants’ visual preferences. We designed a study to contrast young signing versus sign-naïve infants’ looking preferences for different signing avatars. We compare gaze patterns for video stimuli of individual signs produced by (1) a live human signer (Human), (2) an avatar made using computer-synthesized animation (CS Avatar), and (3) an avatar made using high-fidelity motion capture (Mocap avatar). Adult signers recently rated these stimuli for perceptual attitudes (Authors, 2022). The Mocap avatar was rated significantly more favorably than the CS avatar, and adults who learned ASL earlier rated the CS avatar as less favorable than Mocap or Human avatars. Based on these findings, we predict that signing infants are more sensitive to movement quality issues seen in computer-generated avatars than non-signing infants. Eyetracking data will reveal whether the movement quality and appearance of signing avatars matter for infant receptivity of signing input.

Methods: To address early visual looking preferences for such stimuli, we will test hearing children at 6 months (5-7 months) and 12 months (10-14 months). We aim for two contrasts: 6 vs. 12-month-old sign-naïve infants, to examine early nascent visual language preferences. We also contrast 12-month-old sign-naïve infants whose parents use spoken English at home and CODA signing infants whose deaf parents use ASL at home. We predict that preferences will change because of the well-known specialization for home language and increasing socialization by age one year. We aim for 15 infants per group.

Procedure: We will first present each avatar (Human, CS, Mocap; See Figure 1) alone for familiarization. Then, we will present all pairs of avatars side-by-side on a large monitor while the infant is seated on the parent’s lap and measure the infant’s passive gaze with a Tobii 120Hz eyetracker. The stimuli loop for 5 seconds, repeated over several trials, counterbalanced for placement (left, right). Our dependent measure is percent looking time, which reveals which stimulus infants prefer to look at the most throughout the experiment.

Predictions:
1. Main effect of Avatar type as greatest-to-least preference for Human, Mocap, and CS, in that order, if they match adult early signers as reported in Authors, 2022.
2. Younger infants are sensitive to visual phonology, hence they will show discrimination amongst avatars, while older non-signing infants might not because of perceptual narrowing. We will test this with an Age Group x Avatar Type interaction.
3. Modality of language experience alters visual preferences. We will test this prediction via a group difference between signing and nonsigning 12-month-olds.

Implication:
The type of signer that infants choose to look at is likely to be the best stimuli for teaching sign language. Understanding what makes a signing avatar comprehensible and likable will help guide future development in human-computer interaction.

Authors

• Melody Schwenk, Gallaudet University

• Lorna C Quandt, Gallaudet University

• Rain Grant Bosworth, Rochester Institute of Technology
  Presenting Author