Poster #42 - Reliability of Language Samples from Parent-Child Interactions in Toddlers with and without Language Delays

• In Event: Poster Session 05

Abstract

Introduction
Traditional assessments to identify language delays among toddlers rely heavily on standardized tests. However, language sample analysis (LSA) offers a more comprehensive and holistic approach when conceptualizing communication, especially in toddlers with delayed (or limited) language abilities. Most standards for LSA transcription are designed for older (>3 years) and more verbal populations, and the few studies of toddler language focus on typically developing (TD) samples (e.g., Manning et al., 2020) using fully consensused transcripts (e.g., Hadley et al., 2014), and some have not reported reliability estimates. There is a clear need to examine methods for reliably transcribing and analyzing toddlers’ language that yield clinically relevant, ecologically valid measurements of their communication skills. This project examines (1) the interrater reliability of several LSA measures using transcripts from toddlers with and without language delay and (2) the utility of these LSA measures in a young sample with varying degrees of verbal development.

Method
Language samples were transcribed from video recordings of 15-minute naturalistic parent-child interactions obtained at 24 and 36 months among toddlers with significant language delay (LD) or TD from longitudinal studies of early LD as a risk factor for autism spectrum disorder. Transcription conventions were adapted from the standard transcription system Codes for Human Analysis of Transcripts (CHAT). Specifically, additional codes were used for nonword vocalizations and gestures to measure all communicative attempts, not just spoken words. All transcripts were based on consensus between a team of two transcribers. Twenty-one percent of language samples (n=36 total) were transcribed for reliability by two independent teams of transcribers. Reliability estimates were analyzed across age (24mos, n=19; 36mos, n=17) and study group (TD, n= 23; LD, n=13) among LSA measures including exact word agreement, utterance boundary agreement, mean length of utterance (MLU) in words, and mean length of turn (MLT).

Results
Table 1 summarizes descriptive statistics and reliability estimates for each language variable. Overall, reliability estimates were consistently high among the TD group at both ages (ICCs=.902-.997), while there was less consistency among the LD group (ICCs=.527-.995). More conservative measures of exact word agreement yielded lower reliability estimates compared to total words or number of different words. Similarly, MLT (which includes vocalizations, verbalizations, and gestures) was more reliable than MLU (which only includes verbalizations). Significant interactions of age and group were noted for exact word agreement and utterance segmentation (see Figure 1).

Conclusions
Clinicians can obtain reliable LSA measures among toddlers with and without LD, although some caution is warranted for 24-month-olds with LD. For children with LD at 24 months, MLT was more reliable than MLU as toddlers used significantly fewer words and relied on vocalizations or unintelligible word approximations. Since the TD group produced more complex phrase speech at 36 months, compared to the LD group’s production of single words at 24 months, these differences appeared to affect utterance segmentation reliability. This study has important implications for capturing ecologically valid measurements of toddlers’ communication skills, and future research should explore how these LSA measures relate to standardized tests.

Authors

• Danielle Mead, San José State University
  Presenting Author

• Stacy Shumway Manwaring, University of Utah

• Amy Wilder, University of Utah

• Lauren Thompson, Washington State University

• Sarah Houghton, University of Utah

• Oyindamola Adedipe, Ohio State University

• Ellora Mohanty, University of Wisconsin, Milwaukee

• Audrey Thurm, National Institute of Mental Health