The Effect of Item and Examinee Characteristics on Item Score and Response Time on the United States Medical Licensing Examination

• In Event: 32.033 - Frontiers and Issues in Written Tests

Abstract

This study investigates the effects of both item and examinee characteristics on item scores and response times on the United States Medical Licensing Examination® Step 3. Item characteristics include Rasch item difficulty measure, word count, and stimulus material; examinee characteristics include Step 2 CS Spoken English Proficiency score, English as a second language status, and Step 3 total score as a covariate. The sample consists of 9220 examinees’ responses to 1096 items from the USMLE Step 3 multiple-choice questions. Logistic regression and linear mixed-effects modeling techniques were used. One main finding is that although higher word count and presence of pictorial material both significantly increase response time, they do not have a significant effect on the item score.

Authors

• Ren Liu, University of Florida

• Jonathan D. Rubright, National Board of Medical Examiners

• Irina Grabovsky, National Board of Medical Examiners