Examining USMLE Step 1 Feature Usage: A Response Process Validity Study

• In Event: Validity and Instrument Development in Professions Education

Abstract

This validity study explores response process data for USMLE Step 1. It examines how often examination features, such as text highlight and strikeout, are used, as well as how their use impacts Step 1 performance, overall and by medical school location. Using a sample of 1,427 examinees who took Step 1 for the first time from June 2020-May 2021, linear regression models with main and interaction effects were estimated. Overall, highlight and strikeout were used more often and across more questions than other features. Use of some features benefited examinees from US medical schools whereas use of others benefited examinees from non-US medical schools. Results have implications for examination format, examination preparation activities, and test taking strategies.

Authors

• Ni Bei, University of Washington

• Ravi Pandian, National Board of Medical Examiners

• Monica M. Cuddy, National Board of Medical Examiners