Preparing Short-Answer Questions for Use in a Standardized Medical Education Exam

• In Event: Constructing Learner Assessments

Abstract

This paper presents a study that compares multiple-choice questions (MCQs) to short-answer questions (SAQs) for use in a standardized medical education exam. Subject matter experts (SMEs) modified existing MCQs to create 100 SAQs. The SAQs and their MCQ counterparts were administered on an operational Internal Medicine subject exam. A different group of SMEs reviewed SAQ responses and developed scoring rubrics. Item difficulty, discrimination, and response times were calculated. On average, SAQs were more difficult than MCQs. Comparison of item discriminations showed no relationship with item format. SAQs took 28% longer (about 23 seconds) to complete. This study differs from previous work by using large examinee samples, professionally developed materials and examining response times.

Authors

• Janet Mee, NBME
  Presenting Author

• Justin Wolczynski, National Board of Medical Examiners
  Non-Presenting Author

• Amy Morales
  Non-Presenting Author

• Miguel A. Paniagua, American College of Physicians
  Non-Presenting Author

• Polina Harik, NBME
  Non-Presenting Author

• Brian E. Clauser, National Board of Medical Examiners
  Non-Presenting Author