Investigating the Variance Structure of Clinical Decision-Making Questions on a Medical Licensing Examination

• In Event: Issues Related to Certification and Licensure Testing

Abstract

In keeping with the focus on competency-based outcomes, medical licensing examinations utilize scenario-based questions to adequately assess clinical decision-making (CDM) skills. Defined by a varying number of items sharing the same clinical presentation, this item format measures abilities using key features (KF), which are critical steps required to diagnose and treat patients in clinical case scenarios. An item consists of one or more KFs. This research will apply generalizability theory to investigate the variance structure of CDM items by examining all facets (cases, questions, and KFs) over empirical data. Error variance from different sources and generalizability coefficients for different measurement conditions will be calculated. Results will help enhance the reliability of CDM assessments.

Authors

• Isaac Li

• Tsung-Hsun Tsai

• Yi Wang, NBOME

• Qiongqiong Liu, National Board of Osteopathic Medical Examiners