Impact of Position on the Response Speed and Accuracy of Scenario Items (Stage 1, 1:50 PM)

• In Event: e-Lightning Ed-Talk Session 4 - Stage 1

Abstract

This study investigates how item position within scenario-based assessments influences response time and accuracy. Using hierarchical models, we analyzed data from a licensure exam with 499 examinees and three-item scenarios. Results showed a consistent position effect on response time: examinees spent the most time on the first item, less on the second, and slightly more on the third. This pattern likely reflects cognitive orientation, efficiency, and re-engagement. While response time was sensitive to item position, accuracy was primarily influenced by item difficulty. These trends held across high- and low-ability groups. Findings underscore the importance of considering position effects in test design and modeling, particularly for clustered or scenario-based formats.

Authors

• Yi Lu, The Federation of State Boards of Physical Therapy

• Yu Zhang, The Federation of State Boards of Physical Therapy

• Lorin Mueller, Federation of State Boards of Physical Therapy