Advantages of Touch Screen Computer Adaptive Technology for Assessing Science for Low-Income Preschool Populations

• In Event: Advancements in Assessing the Early Educational Experiences of Diverse Children and Families Living in Poverty

Abstract

Objectives
Lens on Science is a project to develop and validate a computer adaptive touch screen assessment (CAT) of young children’s science proficiency. Items were created with science proficiency defined as knowledge in a small set of core, big ideas, acquired during science practices with attention to cross-cutting concepts (NRC, 2012). Items are coded for content, science practice and cross-cutting concept.

Perspective(s) or theoretical framework
Computers offer innovative ways to assess children, with benefits over traditional paper/pencil or one-on-one methods (Wainer, 1990): A CAT estimates level of ability using fewer items than fixed-format counterparts; expensive trainings for assessors are not necessary; rich, comprehensive data, difficult and expensive to capture with traditional formats, are easily obtained; expensive and error prone transfer of data from paper to electronic format are not necessary; examinees receive a unique set of CAT items tailored for that examinee; the CAT item bank can be expanded with additional items over time.

Methods
Item development followed strict guidelines and steps recommended for measure development (e.g., Crocker & Algina, 1986; Osterlind, 2006) that included: (1) establishing the test’s purpose, (2) creating a table of specifications, (3) creating items, (4) review by an expert panel, (5) data on item characteristics from a large, target-population sample, (6) analysis of these data, (7) revising items as necessary, (8) additional data on item characteristics of revised items, (9) follow-up item analyses, and (10) validity data.

Data sources
To assess item characteristics, large samples of multi-ethnic preschool children attending Head Start in a large urban city were administered a random set of CAT items. For validity, a large sample of children from the same program was administered items using the adaptive framework. A subsample received the CAT in the fall and spring along with directly administered language and math assessments at both time points. Teachers also completed behavioral measures on the subsample.

Results
The 400+ items (see Figure 1 for sample item) all have excellent item characteristics (point-biserial values > .20) and cover a wide range of abilities (-3.2 through +3.5). Correlations from the fall data collection were positive and moderate with direct assessments of math and literacy and positive and modest with behavioral measures. Spring data are currently being analyzed and will be included in the presentation.

Scientific or scholarly significance
Young children, especially those from diverse backgrounds, have missed important early learning opportunities in which their natural curiosity in science could serve as cognitive models for the challenging years of schools that lie ahead. National, state, and local efforts focused on early childhood science education is providing a major opportunity to invigorate early childhood curricula with the excitement for learning and exploration that science provides. Reliable, valid and child friendly assessments are critically needed to accurately evaluate the impact of these science practices on young children’s science proficiency. Follow-up work has begun developing Enfoque En Ciencia, a parallel, equated Spanish version of the Lens assessment valid for use with Latino preschool children using the same CAT platform developed for the Lens project.

Author

• Daryl B. Greenfield, University of Miami