Profiles of Learning in the 2009 NAEP Science Interactive Computer Task

• In Event: 35.020 - Excellence in Education Research: Early-Career Scholars and Their Work

Abstract

In 2004, the National Assessment Governing Board (NAGB) proposed a new science framework (Champagne et al., 2004) that was eventually implemented in 2009 (US. Department of Education) which proposed to bring together advances in cognitive science and measurement theory to measure abilities such as scientific inquiry skills. One of these specifications was the development and administration of the 2009 National Assessment of Educational Progress (NAEP) science interactive computer tasks (ICTs).

The NAEP 2009 Science ICTs contain 20 or 40 minute tasks designed to assess how well students can perform scientific investigations, draw valid conclusions, and explain their results. These skills can be assessed through POE item sets (NAEP Framework 2009, p.124):

• Predict. Students provide a prediction for what might happen in a real-world science situation
• Observe. Students conduct an investigation and observe what happens
• Explain. Students explain what they have observed by interpreting data or drawing conclusions

Operationalization of learning effect in the Science ICT context
The assumption of learning is based on principles of formative assessment where the primary concern is with how the quality of student responses can be used to shape and improve students’ competence (Sadler, 1989). Feedback is a key element in formative assessment, with the intended goal of altering the achievement gap between actual and reference levels. Creating profiles of achievement in science inquiry allows teachers to assess the quality of student performance, and allow for student self-monitoring. The goal is to allow students in low action levels to match reference level performance. Students with correct prior knowledge are presumed to be closer to the reference performance level, whereas students with lower prior knowledge coming into the assessment, although have a larger gap to close, have the most potential to exhibit learning effects. The operationalization of learning in this context is defined as the 1) potential between actual and reference achievement levels, and 2) the amount of actual and reference closure.

Author

• Johnny Lin, Educational Testing Service