Assessing Students' Progress on the Energy Concept Using Three-Dimensional Assessments

• In Event: 47.088 - Overcoming Challenges in Developing and Implementing Next Generation Science Standards–Aligned Instructional Materials and Assessments

Abstract

The Next Generation Science Standards (NGSS Lead States, 2013) calls for instruction in science that fosters an integrated understanding of science and engineering practices, crosscutting concepts, and disciplinary core ideas. Along with this new approach to instruction, new assessments are being called for to assess this vision of integrated, three-dimensional science learning. The purpose of this project is to develop assessments that can measure students’ ability to use these three dimensions together to make sense of energy-related phenomena.

Assessment development takes a construct-centered approach that draws from frameworks such as Evidence Centered Design (Mislevy et al., 2003) and Construct Modeling (Wilson, 2005) as well as our own previous work. Performance expectations (PEs) related to elementary, middle, and high school energy ideas were identified to define the specific practices, crosscutting concepts, and core ideas to be assessed. We unpacked the relevant PEs into finer-grained statements that explicitly state how students should use the practices in the context of the crosscutting concepts and energy core ideas to make sense of real-world phenomena. Then we drafted scenario-based tasks that are made up of interrelated multiple-choice and constructed-response items as recommended by the National Research Council (NRC, 2104). Thirty-five draft tasks were pilot tested with over 5,000 students in grades four through twelve from across the U.S. The findings from the pilot test are being used to inform rubric development and item revision. Additionally, we are conducting a study to compare the inferences that can be made from the results of multiple-choice and constructed-response versions of items to improve the cost-effectiveness of the tasks.

During task development we encountered a number of challenges that we hope to discuss during the panel session. These challenges include: (1) selecting engaging phenomena that are explainable by grade-appropriate ideas, practices, and crosscutting concepts, (2) more clearly defining the crosscutting concepts and determining how to best assess this dimension, (3) effectively incorporating the three-dimensions into the tasks to ensure that the students’ are able to provide evidence that they can use the dimensions together to make sense of phenomena, (4) determining the appropriate level of scaffolding to include within a task so that the tasks are accessible to a wide range of students, (5) finding the right balance of item formats to maximize the cost effectiveness of the tasks, and (6) developing practical yet effective scoring rubrics to provide users with information about students’ three-dimensional understanding that can inform instruction. Additionally, because the disciplinary core idea focus of our assessments is energy, we are interested in discussing how appropriate it is to use energy as the crosscutting concept as well.

Authors

• Cari F. Herrmann-Abell, BSCS Science Learning

• Joseph Hardcastle, American Association for the Advancement of Science

• George E. DeBoer, American Association for the Advancement of Science