Multidimensional Item Response Theory Application: The Validity of Internal Structure of a Developmental Measure

• In Event: Poster Session 09
  In Poster Session: PS 09 Topic - Methods, History, Theory

Abstract

Part C of the Individuals with Disabilities Education Act (IDEA, 2004), a federally authorized service delivery program, supports states to provide early intervention services for infants and toddlers ages birth through 36 months who have, or are at risk for, developmental delays or disabilities. In 2018, the snapshot count of children served in Part C Early Intervention (EI) programs was over 800,000, representing 3.5% of the U.S. population of children birth through 2 years of age (Office of Special Education Programs, 2020).
To determine eligibility to participate in Part C EI services, states must assess the developmental functioning of referred children in five domains: cognitive, adaptive, communication, social-emotional, and physical. Meanwhile, for accountability purposes, states are required to report annually the percentage of infants and toddlers exiting Part C EI programs who show progress in three outcome areas: social-emotional skills, acquisition and use of knowledge and skills, and use of appropriate behavior to meet their needs. The Early Childhood Technical Assistance (ECTA) Center, a center funded to support IDEA early childhood programs, refers to the outcome areas as “functional outcomes” since the primary aim is to focus on functioning rather than domains (ECTA Center, 2020).
The accountability requirements for Part C EI programs do not mandate the use of any specific instrument to ground states’ reports of the percent of children who make progress in each outcome area. Several states utilize pre-post data from the Battelle Developmental Inventory, 2nd Edition (BDI-2; Newborg, 2005). The challenge for states that use BDI-2 data is whether, and if so how, to combine results from the BDI-2’s five developmental domains to report children’s progress in the three federally defined outcome areas. Thus, the purpose of this study is to assess the validity of BDI-2’s developmental measures using different partial credit models (PCMs).
In our study, we fit four PCMs to BDI-2 assessment data from a de-identified sample of children (n = 2984) ages 24 through 36 months obtained from a large local Part C EI program in the southeastern U.S. We limited the sample to assessments conducted in the 24- to 36-month age range because BDI-2 domain scores for children under 24 months are based on a smaller number of subdomains than are scores for children at least 24 months of age. The four PCMs tested were 1) one unidimensional model, 2) five unidimensional models, 3) a multidimensional model with 5 domains as factors, and 4) a multidimensional model with 3 outcome areas as factors, defined as (i) only the personal social domain, to capture social-emotional skills; the cognitive and communication domains combined, to capture acquisition and use of knowledge and skills; and (iii) the adaptive and motor domains combined, to capture use of appropriate action to meet needs. The results indicated that the multidimensional model with five domains as factors was the best fitting model to the data. This finding, which supports the original measurement structure of the assessment, calls into question the suitability of combining domains for accountability purposes.

Authors

• Seniz Celimli Aksoy, Frostburg State University
  Presenting Author

• Batya Elbaum, University of Miami
  Non-Presenting Author

• Jue Wang, University of Miami
  Non-Presenting Author