Assessing the Upper Elementary Next Generation Science Standards Engineering Performance Expectations Independently of the Natural Science Disciplines

• In Event: Transfer for Learning in the Classroom: Developing Contexts to Examine STEM Learning Skills and Strategies

Abstract

K-12 engineering education is gaining traction in the U.S. largely because of its inclusion in the Framework for K-12 Science Education (NRC, 2012) and in the Next Generation Science Standards (NGSS) (NGSS Lead States, 2013). The Framework describes engineering as involving the application of science disciplinary to engineering solution of engineering problems. As such, assessing the NGSS engineering performance expectations (PEs) presents particular challenges. Although students typically engage in engineering alongside a natural science discipline (Earth, life, or physical science), stakeholders may wish to assess the extent to which students can generalize their engineering proficiency beyond a specific science discipline.

We use evidence-centered design (ECD) (Mislevy & Haertel, 2006) to guide the design of science-independent NGSS-aligned engineering assessments for the upper elementary grade band. ECD helps structure the connections among the targeted student proficiencies, task design features, and observable evidence. Assessment designers can use ECD to demonstrate how particular student performances provide evidence for students’ proficiency on particular constructs. In this work we build on an established approach for designing NGSS-aligned assessment tasks in physical and life science (Author, 2016).

Our design process consists of four elements:
1) Unpacking entails articulating the boundaries and structure of the domain to be assessed, including identifying key aspects of the DCIs and practices, defining expectations for understanding at the target grade band, determining upper proficiency boundaries, and identifying prerequisite knowledge.
2) We articulate assessable performance statements for PEs that are too broad to assess using a single assessment task scenario. We decompose these PEs into multiple, distinct performance statements and align tasks and rubrics to these statements, which reflect the key aspects of the engineering domain as identified in our unpacking.
3) For each performance statement, we determined task design specifications that provide the basis for aligning task and rubric design to a performance statement: (1) evidence statements, which describe features of student responses that constitute evidence of proficiency, (2) characteristic task features, which must be included in a task to elicit the target proficiency, and (3) variable task features, which can shift the difficulty or focus of a task.
4) Based on the design specifications, we developed tasks and rubrics that can be used to support claims about students’ NGSS-aligned engineering proficiency in an everyday problem context. The tasks employ a science-independent problem context accessible to elementary students that does not require natural science disciplinary content knowledge. We designed rubrics to align with evidence statements.

On early design iterations of the assessment tasks, we computed descriptive statistics and conducted reliability analyses. This preliminary validity evidence is consistent with each task measuring a different aspect of students’ engineering proficiency (e.g., defining problems, generating solutions, comparing solutions). This poster will present analyses on student responses collected in the Spring of 2019. The findings will inform the design of high-quality NGSS-aligned engineering assessments that are needed to measure student learning outcomes from NGSS-aligned engineering instruction and evaluate NGSS-aligned engineering curriculum interventions.

Authors

• Kevin McElhaney, Digital Promise

• Satabdi Basu, SRI International

• Elizabeth McBride, SRI International

• Reina Fujii, SRI International

• Jennifer L. Chiu, University of Virginia

• Anne M McAlister, University of Virginia