The Criterion-Zone Boundary Approach to Studying Progress Toward Reaching Statewide Early Learning Goals

• In Event: 81.028 - Supporting Children's Early Learning and Kindergarten Goals With a Standards-Aligned Assessment: Effective Practices in Statewide Aggregate Reporting

Abstract

1. Objectives or purposes
To describe the process for relating the results of California’s developmental assessments, the DRDP (2015) to standards-based expectations for children’s development from infancy through kindergarten, thereby determining ‘criterion-zone boundaries’ (CZB) for developmental readiness.

2. Perspective(s) or theoretical framework
CZBs demarcate readiness zones that align with developmental expectations specified in the state’s standards/foundations. The process of determining criterion-zone boundaries involved the work of multiple stakeholders (policymakers, researchers, educators, etc.). The work of these stakeholders, and our understanding of the results and utility of the readiness zones, is informed by a coalescence of developmental theory, alignment with foundations/standards, multidimensional item response theory (MIRT), and other forms of empirical evidence.

3. Methods, techniques, or modes of inquiry
The method for establishing and using CZBs was multi-step.
1) Child development research experts and teachers reviewed the correspondence between the assessment instruments and the foundations/standards.
2) After intensive training, the child development research experts used an item mapping process called Construct Mapping to make criterion-zone boundary recommendations at each key age in different domains of the foundations and the kindergarten standards.
3) The boundary recommendations resulted in seven readiness zones: 0-8 months, 8-18 months, 18-36 months, 36-48 months, 48-60 months, 60 months – end of kindergarten, and beyond the end of kindergarten.
4) DRDP data derived from a multidimensional IRT measurement model were analyzed to describe the proportion of children in each of the seven readiness zones.

4. Data sources, evidence, objects, or materials
The team used the DRDP (2015) instruments and California’s foundations/standards to establish the readiness zones (i.e., CZBs). Next, psychometricians used scores DRDP data collected by teachers in state-funded early childhood education, transitional kindergarten, kindergarten, and after school programs between Fall 2015 and Spring 2017 for multidimensional IRT (MIRT) analysis followed by CZB analysis.

5. Results and/or substantiated conclusions or warrants for arguments/point of view
The results from this process presents the proportion of California children in this sample who are in each readiness zone for each developmental domain and across developmental domains. Readiness can be tracked over time and for specific subsamples (e.g., English learners or students who receive special interventions).

6. Scientific or scholarly significance of the study or work
The readiness zones are the product of a developmentally-informed alignment process and the MIRT measurement model. The process allows DRDP (2015) data to be translated into criterion-based, psychometrically substantiated readiness zones. This is novel both in the assessment literature and in the practical applications of aggregating readiness assessment data in early childhood. Comparisons can be made across ages, subsamples (e.g., dual language learners), and academic terms, with both longitudinal and cross-sectional approaches, allowing the state to reflect on how well California’s children are supported and prepared for kindergarten throughout infancy, toddlerhood, and preschool. This information can then inform the state’s efforts to support early learning and school readiness for all of California’s children.

Author

• Emily Newton, WestEd