The Impact of Item, Test, and Test-Taker Characteristics on Engagement During NAEP Assessments

Session Type: Symposium

Abstract

NAEP assessments serve a vital purpose by measuring U.S. students’ proficiencies in multiple domains, but they are low stakes for the students that complete them. Low test-taking engagement (TTE) has been found to occur for low-stakes assessments and can be a threat to outcome validity. Prior research has focused primarily on identifying low TTE and methods to filter those responses to improve outcome validity. The three papers in the present symposium adopt a different approach. Each paper investigates how test design decisions (e.g., item features, test assembly) and test-taker characteristics (e.g., proficiency level, opportunities to learn) interact to better understand how tests can be designed to minimize the occurrence of low TTE in a proactive manner.

Sub Unit

• SIG-NAEP Studies

Chair

• Blair Lehman, Educational Testing Service

Papers

• Strengthening NAEP Interactive Computer Tasks Through Increased Student Engagement - Kristin Stoeffler, ACT, Inc.; Yan Wang, American Institutes for Research; Emmanuel Sikali, U.S. Department of Education

• Impact of Test Design Decisions on Test-Taker Engagement - Blair Lehman, Educational Testing Service; Jonathan Steinberg, EurekaFacts; Jesse R. Sparks, Educational Testing Service; Jennifer Bailey, ETS; Fred Yan, Educational Testing Service; Jung Aa Moon, Honeywell

• Comparing Data-Driven and Theory-Driven Methods of Detecting Engaging in Reading Assessment - Jesse R. Sparks, Educational Testing Service; Jonathan Steinberg, EurekaFacts; Blair Lehman, Educational Testing Service; Burcu Arslan, Educational Testing Service Global; Jennifer Bailey, ETS

Discussant

• Nadia McLaughlin, U.S. Department of Education