BackHome
Refresh: OffView Personal Schedule
Individual Submission Summary
Share...

Direct link:

Poster #110 - Hearing Babies Respond to Language's Patterning and Social Contingency with a Signing Avatar: Acquisition Insights

Fri, March 22, 2:30 to 3:45pm, Baltimore Convention Center, Floor: Level 1, Exhibit Hall B

Integrative Statement

A novel learning tool with artificial agents was built to facilitate language learning in young babies deprived of language exposure during a critical period in child development, 6-12 months. Here, babies have peaked sensitivity to the rhythmic temporal patterning of language, which permits segmentation, categorization, and discernment of statistical regularities of phonetic-syllabic units central to word/language learning, and early reading success1. One target population are the many deaf babies with minimal usable access to language in early life, hence our use of a natural signed language, American Sign Language (ASL). The system, “RAVE” (Robot, AVatar, thermal Enhanced language learning tool) involves an embodied robot (directs babies’ attention to an avatar screen), avatar (produces language/Linguistic Nursery Rhymes in ASL; routine Social Communications such as HI, BYE-BYE; non-linguistic/non socially-contingent body movements/postures, “Idle”), and thermal imaging (an innovation that permits detection of Autonomic Nervous System activity associated with emotional engagement and attention);2 the babies’ nature/degree of engagement provided an index of when they were “ready to learn,” which then triggered the avatar when to start and cease a socially contingent communication. Dialogue scripts then guided selection of avatar communications with baby that were socially contingent on the baby’s states of engagement and productions. Earlier studies suggest that babies can follow a robot’s gaze3, but infant language learning from a TV screen is problematical4. Our scientific challenge and question were whether babies can detect internal differences among the avatar’s communicative modes by producing differential behavioral responses; the hypothesis being that differential behavioral responses would provide evidence of a potential to learn language from an artificial agent. METHOD. Experiment w/8 babies: 1 deaf/sign-exposed, 7 hearing: 1 sign+speech exposed, 6 non-sign exposed (a design feature to reveal salient linguistic perception/production features independent of language meaning); 7-13mths; seated before RAVE on parent’s lap (Fig 1). A key feature of the ASL stimuli was that they contained the rhythmic temporal patterning universal to all languages and critical to this age1;5. RESULTS. All 8 babies produced differential behavioral responses to the avatar’s different communicative modes/conversational turns (i.e., sustained visual attention, social gestures, and linguistic). Case studies of 4 showed largest percentage of their linguistic responses to the avatar’s Linguistic Nursery Rhymes: 36% linguistic responses to avatar’s Nursery Rhymes vs 26 % to avatar’s Idle, 26% to avatar’s Social Communications, 24% during 3-Way avatar, baby, robot exchanges. Babies’ linguistic responses included manual babbling, production of proto-sign phonetic units, proto-signs, linguistic sign-phonetic, sign imitations. DISCUSSION: The findings are remarkable because most of these hearing babies did not understand the meaning of ASL and instead appeared riveted by the universal rhythmic temporal patterning of language; add, the avatar was on a TV screen. Beyond social interaction, babies responded greatest to the avatar when its productions were socially contingent (e.g., Nursery Rhymes, Social Gestures versus Idle), thus providing powerful insights that the presence of language’s rhythmic patterning and social contingency constitute two potent and necessary features of human language acquisition. The study demonstrates the potential for language learning from agents in young babies.

Authors