An Online Computerized Adaptive Test (CAT) of Children's Vocabulary Development in English and Mexican Spanish

• In Event: Poster Session 12
  In Poster Session: PS 12 Topic - Language, Communication

Abstract

Measuring the growth of young children's vocabulary is important for researchers seeking to understand language learning as well as for clinicians aiming to identify children at risk for early deficits. The MacArthur-Bates Communicative Development Inventory (MCDI, Fenson et al. 2007) is a set of reliable and valid parent-reported vocabulary checklists that are widely used for assessing language from 8-18 months (Words and Gestures; WG), 16-30 months (Words and Sentences; WS) and 30-36 months (CDI-III). Although the long form versions of the CDIs have many advantages, one clear drawback is the length of the checklists (400-600 words plus grammatical and other items) and the corresponding time required for parents to complete the forms. Moreover, because the word lists are designed to be appropriate for children across a broad range, some words may not be known by younger children. Although these forms are a “gold standard” for measuring early language (Bornstein et al., 2012), they are time-consuming to administer, thereby limiting their usefulness in many clinical and research contexts.

Computer-adaptive testing (CAT) techniques hold the promise of dramatically decreasing CDI completion time by administering only the most relevant and diagnostic items. One simulation study showed that even a 50 item CAT CDI constructed using Item Response Theory (IRT) models could recover highly accurate percentile scores for children acquiring English (Makransky et al., 2016). Here, we build on this work using a larger dataset which extends across forms to create even shorter CAT CDI forms that are appropriate for a broader age range and that are appropriate for English as well as Mexican Spanish.

We used data from Wordbank (Frank et al., 2017) for 7,633 English-speaking children aged 12-36 months and 1,692 Spanish-speaking children aged 12-30 months, across WS, WG, and CDI-III. We compared 4 different logistic IRT models (Rasch, 2PL, 3PL, 4PL) using standard model comparison criteria and found that 2-parameter (2PL) logistic model fits best for the majority of the 680 pooled items across all three instruments (Table 1).

We then fit these 2PL models for both English and Spanish, using all three datasets to create a CDI CAT to assess vocabulary production for children ages 12 - 36 months and for vocabulary comprehension for ages 8 - 18 months. We conducted real data CAT simulations on the Wordbank dataset, administering simulated tests of varying length (25-400 items). Results indicated that even 25-item CATs estimate participant abilities very well, with correlations of r=.99 with full scores (Table 2). Further, we find that these short CATs work well with relatively little bias across the full age range, suggesting that CAT can be used in place of the CDI to provide a fast estimate of children's vocabulary without compromising accuracy and precision. We provide our item bank along with fitted parameters, offer recommendations for how to run a CAT version CDI, and suggest when CDI CAT may be inappropriate. These models are currently implemented in an online platform and an empirical validation study (n=200) comparing CDI CAT to long form performance is ongoing.

Authors

• George Kachergis, Stanford University
  Presenting Author

• Virginia A. Marchman, Stanford University
  Non-Presenting Author

• Philip S Dale, University of New Mexico
  Non-Presenting Author

• Henry Mehta
  Non-Presenting Author

• Jessica Mankewitz, Stanford University
  Non-Presenting Author

• Michael C. Frank, Stanford University
  Non-Presenting Author