Optimizing the Number of Distractors via Advanced Item Analysis

• In Event: Measurement Methodology Innovation

Abstract

Analyzing distractor qualities is vital in developing high-quality items,yet traditional approaches focus on key answers only and are less informative. The number of distractors can be parsimonized to improve the item development. 44332 examinees to nineteen multiple-choice questions were analyzed, while the distractor qualities were evaluated via both traditional methods and the proposed one utilizing canonical correlation index within a simulation mechanism. The traditional item analysis results showed that most items had acceptable psychometric properties, and two items were flagged for low item difficulty and discrimination. The proposed method revealed that about one-third of items had poorly functioning distractors and a small effect size of distractor discrimination. The simulation-based decision study confirmed that shrinking distractor numbers to four was viable.

Authors

• Zhehan Jiang, Peking University
  Presenting Author

• Qianqian Pan, Nanyang Technological University
  Non-Presenting Author

• Yuting Han, Beijing Language and Culture University
  Non-Presenting Author

• Jinying Ouyang, Peking University
  Non-Presenting Author

• Lingling Xu, Peking University
  Non-Presenting Author

• Dexin Shi, University of South Carolina
  Non-Presenting Author

• Ren Liu, University of California - Merced
  Non-Presenting Author