Objective (Automated) Measurement of Infant and Child Behavior: Reliability, Validity, and Discovery

Session Type: Paper Symposium

Abstract

Infants explore and engage with the social world in the first months of life using facial expressions, gaze, and vocalizations. The systematic classification and tracking of affective facial expressions and vocalizations has been invaluable for studying typical and atypical development across a wide range of early socio-cognitive and socio-emotional processes, including visual attention, preverbal communication, and self-regulation. The time commitment involved in manual coding, however, exerts a strong downward pressure on infant sample sizes. Progress has been made in the development of automated measurement of infant behavior, offering the potential for rich and reproducible results at lower time-investment than manual coding. This symposium discusses state-of-the-art applications of automatic techniques to: 1) classify affective facial expressions from a longitudinal study of infants at 4 and 8 months of age during face-to-face interactions with mother, father, and unfamiliar adult using a commercial tool (Baby FaceReader 9), 2) measure Duchenne and non-Duchenne smiling and cry-faces during face-to-face/Still-face interactions of 4-month old infants using an open source tool (Infant AFAR), 3) demonstrate an application of automatically measured affective facial expressions, gaze, and vocalizations to predict symptoms of Autism Spectrum Disorder symptomatology using a combination of open-source and commercial tools. The discussant has made innovative, long-lasting, and influential contributions to the field of multimodal interaction that span automatic detection of facial expression, multimodal mother-infant communication, and multimodal communication of psychopathology. The discussant will critically address the knowledge we can gain from and the challenges encountered in the application of automated tools.

Sub Unit

• Social, Emotional, Personality

Chair

• Martina Zaharieva, University of Amsterdam

Discussant

• Jeffrey Cohn, University of Pittsburgh

Individual Presentations

• Automated Measurement of Affective Facial Expressions in 4- and 8-Month-Olds: Reliability with Manual Coding - Presenting Author: Martina Zaharieva, University of Amsterdam; Eliala Alice Salvadori, University of Amsterdam; Daniel Messinger, University of Miami; Ingmar Visser; Cristina Colonnesi, University of Amsterdam

• Automated Measurement of Infant and Mother Duchenne Facial Expressions in the Face-to-Face/Still-Face - Presenting Author: Yeojin Amy Ahn, University of Miami; Onal Ertugrul Itir, University of Utrecht; Sy-Miin Chow, The Pennsylvania State University; Jeffrey Cohn, University of Pittsburgh; Daniel Messinger, University of Miami

• Objective Measurements of Children’s Gaze and Vocal Behavior Associated with Clinician-Rated Autism Symptoms - Presenting Author: Jacquelyn Michele Moffitt, University of Miami; Lynn K Perry, University of Miami; Yeojin Amy Ahn, University of Miami; Jennifer Durocher, University of Miami; Daniel Messinger, University of Miami