A Comparison of Three-Option and Four-Option Multiple-Choice Questions for In-Training Examinations

• In Event: Roundtable: Assessment

Abstract

The objective of this proposal is to evaluate whether and how psychometric properties are changed when the same items were administered as 3-option in comparison to 4-option formats. There were 136 and 146 common items for subspecialty 1 (np2019 = 153, np2020 = 123) and subspecialty 2 (np2019 = 113, np2020 = 132) administered in 2019 and 2020, respectively. No significant differences in item difficulty and discrimination were found between the two formats for either of the 2 subspecialties. The percent-correct scores were significantly higher and test administration time was significantly shorter in 3-option format than those in 4-option format for subspecialty 1; no such differences were found for subspecialty 2. Implications of implementing 3-option format are discussed.

Authors

• Dandan Chen, The American Board of Anesthesiology

• Huaping Sun, The American Board of Anesthesiology