The Impact of Modifying Drug Class Information on Item Performance on a Medical Licensure Exam

• In Event: The Parker Hotel Roundtable 5
  In Roundtable Session: Applied Psychometric Analysis Roundtable

Abstract

The current item development processes for a large-scale medical licensure exam series demand that, if changes are made to a live item, the item must be re-pretested. As medications are withdrawn from the market or recommended medications change, items that refer to these medications must be updated. We examined the differences in response distributions and response durations between 44 original multiple-choice items and 120 successors, where successors were created by changing the drug name in the item vignette, correct response, incorrect response, or some combination of these. Results indicate that item characteristics are significantly affected when the drug substitution occurs in the item’s key, with no other changes resulting in significantly different performances. Implications for item development are discussed.

Authors

• Christopher Runyon, The University of Texas - Austin

• Kimberly A. Swygert, National Board of Medical Examiners

• Steven Arthur Haist, National Board of Medical Examiners

• Danielle McSweeney, National Board of Medical Examiners

• Susan Carracappa, National Board of Medical Examiners