How Do Test-Takers Perceive Leveling in Scenario-Based Tasks? An Initial Exploration

• In Event: Marriott Roundtable Session 17
  In Roundtable Session: The Integration of Cognition and Assessment: Improving Assessment Development and Score Interpretations and Utilization

Abstract

Multi-item scenario-based tasks offer students the ability to demonstrate their knowledge and skills in realistic situations, but can lead to measurement difficulties as the assumed independence between items is violated. Leveling (or resetting) is often utilized to break the dependence between items by providing all test-takers with the correct information, regardless of prior answer accuracy. However, there is limited understanding of how the use of leveling is perceived by test-takers and how it impacts the test-taking experience. The present paper serves as an initial exploration of how leveling is perceived in an effort to understand how test-takers utilize the information they are provided.

Authors

• Blair Lehman, Educational Testing Service

• Devon Kinsey, Educational Testing Service

• Bridgid Finn, Educational Testing Service