Enfoque en Ciencia: Assessing Science Competence in Latino Preschoolers by Advantaging the Affordances in Technology

• In Event: 2-185 - Measuring School Readiness Skills of Dual Language Learners: Development and Validation of Assessment Tools

Integrative Statement

A computer adaptive assessment (CAT) offer benefits over traditional one-on-one paper methods for assessing young children with limited attention spans: A CAT estimates level of ability using fewer items than fixed-format counterparts; expensive trainings for assessors are not necessary; rich, comprehensive data, difficult and expensive to capture with traditional formats, are easily obtained; expensive and error prone transfer of data from paper to electronic format are not necessary; examinees receive a unique set of CAT items tailored for that examinee; the CAT item bank can be expanded with additional items over time.
Methods
Lens on Science an English administered CAT of preschool children’s science competency, was developed following strict guidelines and steps recommended for measure development that included: (1) establishing the test’s purpose, (2) creating a table of specifications, (3) creating items, (4) review by an expert panel, (5) data on item characteristics from a large, target-population sample, (6) analysis of these data, (7) revising items as necessary, (8) additional data on item characteristics of revised items, (9) follow-up item analyses, and (10) validity data. Lens on Science is currently being used in multiple studies across the country to develop and evaluate preschool science learning.
There is a large and expanding populations of Latino, Spanish language dominant preschoolers. These children are often excluded from studies on preschool science learning as they are unable to meet English proficiency screening for the Lens on Science assessment. Enfoque en Ciencia was developed to address this concern.
When developing assessments for children from culturally and linguistically diverse backgrounds it is important to consider the heterogeneity in the population. Such heterogeneity has implications for the development of assessment items, both in terms of language and cultural load. Another important issue to recognize when translating items is that the act of translating an item can alter the psychometric properties (e.g., difficulty and discrimination) of an item, which in turn impacts the interpretation of the scores generated by an assessment. To address these concerns, the development of Enfoque en Ciencia included along with the steps listed above used to develop Lens on Science (a) a rigorous content review for linguistic and cultural equivalence of the items to those of the original language form; (b) a complete examination of the measurement equivalence of translated items via the framework of differential item functioning; and (c) a formal equating process that places the scores of the translated assessment on the same scale as Lens on Science.
Data sources
To assess item characteristics, large samples of multi-ethnic preschool children attending Head Start in a large urban city were administered a random set of CAT items. For validity, a large sample of children from the same program was administered items using the adaptive framework. A subsample received the CAT along with directly administered language and math assessments in Spanish. Teachers also completed behavioral measures on the subsample.
Results
The presentation will include information on the translation process, evaluation of item characteristics and dialect bias and results from the validity study.

Authors

• Daryl Greenfield, University of Miami
  Presenting Author

• Randall Penfield, University of North Carolina
  Non-Presenting Author

• Ximena Dominguez, Digital Promise
  Non-Presenting Author

• Sandra Barrueco, The Catholic University of America
  Non-Presenting Author