Innovation in Evaluation: Creating a Science Assessment for Kindergarten Students

• In Event: Integrating Literacy and Science in Kindergarten: Results From a Large-Scale Mixed-Methods Evaluation

Abstract

Objectives or purposes: The Zoology One study underwent a two-year process to develop an instrument to adequately measure the growth in science learning that evaluators hypothesized they would see as a result of a science-infused literacy curriculum in kindergarten.

This paper will:
• Explain the team’s consideration of whether to use an existing assessment; modify an assessment; or develop a new assessment.
• Detail the process of developing, piloting and evaluating items;
• Share our experience navigating the challenges of assessment development given the constraints of the district and schools.

Perspective(s) or theoretical framework: Published reports from program evaluations generally address the choice of assessment instruments only in passing. Assessments are identified, and there may be a brief discussion of the rationale for particular decisions. In contrast, this paper addresses the nuanced and complex issues our team has encountered with making these decisions. Assessing young students in an urban school district provides a set of unique challenges. Creating instruments in the context of program evaluation introduces more challenges still. All of these topics will be of interest to program evaluators, and will be addressed in this paper.

Methods, techniques, or modes of inquiry: The assessment that resulted from the process described in this paper, the Early Life Sciences Assessment, or ELSA, was created with an urban kindergarten sample in mind. It was developed to measure a range of science content knowledge with questions that kindergarten students would easily be able to understand. The instrument is meant to isolate science content knowledge and avoid an indirect measurement or conflation with ability to read or give a verbal response. The processes used to determine the most critical features of this new assessment, and the process for finding the most suitable design given the goals of the instrument will be detailed in the paper.

Data sources, evidence, objects, or materials: This paper explains the process and decisions made around this assessment, as informed by:
• Science Learning Assessment (Samarapungavan, Patrick, & Matzicoupolis, 2006)
• Next Generation Science Standards (http://www.nextgenscience.org, 2018)
• Standards for Educational Psychological Testing (American Educational Research Association, 2014)
• Item Response Theory (Raykov, 2018)

Results and/or substantiated conclusions or warrants for arguments/point of view: In the context of our evaluation, our small team sought an adequate tool to measure science learning in urban kindergarten students. Although the original proposal for this evaluation did not include the creation of a new instrument, it became apparent in the first year of the study that a large amount of resources would need to be dedicated to just that in the second year. This paper will present the benefits and limitations of this approach.

Scientific or scholarly significance of the study or work: This paper seeks to serve as a reference to others who intend to develop academic assessments for young students as part of an evaluation. Our experience in this area will provide a much-needed perspective on the considerations required when developing assessments for young children in the context of large evaluations.

Author

• Tesla DuBois, University of Pennsylvania