Development and Validation of the School Readiness Curriculum Based Measurement System

• In Event: 2-185 - Measuring School Readiness Skills of Dual Language Learners: Development and Validation of Assessment Tools

Integrative Statement

Purpose
Early identification and monitoring children's learning are important features of programs that help close the achievement gap. Unfortunately, many existing preschool measures have limited reliability, validity, and utility. We designed fifteen item pools to assess the language and literacy skills of young children who speak English and/or Spanish. Vocabulary, blending, letter name knowledge, and graphophonemic knowledge are assessed in English and Spanish. Rhyming skills are assessed only in English. We highlight how our development procedures yielded tests with unique advantages over existing assessment tools. And, we report psychometric evaluations of four assessment batteries created from these item pools, including English and Spanish versions of the School Readiness Curriculum Based Measurement System (SRCBM) and English and Spanish versions of Texas Kindergarten Entry Assessment (TXKEA).
Method
To minimize floor and ceiling effects that plague existing progress monitoring tools in early childhood, each construct was measured by a single task with broad item content or by multiple tasks that provide broad coverage of the ability continuum. To maximize sensitive to learning, large numbers of items were created all along each ability continuum. Item pools were scaled on thousands of 3- to 6 year-olds (ns=1500-4100) from 97 schools in Texas and Florida. Sensitivity and efficiency were maximized by deleting items with low discriminations. Test bias was minimized by deleting items that demonstrated differential item functioning by gender, ethnicity, or English language status. The breadth and depth of construct coverage along with large scaling samples permitted creation of both leveled short forms and parallel short forms that are brief and easy for educators to administer yet sensitive to children’s growth and development. Monte Carlo methods were used to generate hundreds of thousands of item combinations of realistic form lengths, and then parallel forms were selected based on test reliabilities, similarity of test information curves, variability in gaps between difficulties of adjacent items, number of overlapping items, and face validity. Ultimately, we created 40 short forms for SRCBM and 8 short forms for TXKEA, most of which ranged from 8 to 11 items in length.
Four longitudinal studies were conducted to validate English SRCBM, Spanish SRCBM, English TXKEA, and Spanish TXKEA. Participants included 381 L1 English speaking 3- to 6-year-olds, 294 L1 Spanish speaking 3- to 6-year-olds, 215 L1 English speaking kindergarteners, and 234 L1 Spanish speaking kindergarteners, respectively. Participants completed the new measures and standardized criterion measures at multiple time points during the school year.
Results
Cronbach’s alphas ranged from .71 to .91. Concurrent convergent validity coefficients ranged from .45 to .95. Predictive validity coefficients ranged from .52 - .87. Linear slope parameters were significant in all latent growth curve models.
Conclusions
Most SRCBM and TXKEA subtests demonstrated excellent reliability, validity, sensitivity, and predictive utility when one considers their test lengths, test formats, age of participants, and the status quo of similar measures in early childhood education. Teachers and researchers will be able to use the SRCBM to assess 3- to 5-year-olds' language and literacy skills in English and Spanish.

Authors

• Jason L. Anthony, University of South Florida
  Presenting Author

• Jeffrey M. Williams, University of South Florida
  Non-Presenting Author

• Matthew Foster, University of South Florida
  Non-Presenting Author

• Janelle J. Montroy, University of Texas Health Science Center, Children's Learning Institute
  Non-Presenting Author

• Michael Assel, University of Texas Health Science Center at Houston
  Non-Presenting Author

• Noe Erazo, University of South Florida
  Non-Presenting Author