Validity Evidence for Yes-No Angoff Standard-Setting in a Preclinical Medical School Curriculum

• In Event: Exploring Learning, Evaluation, and Assessment in Professions Education

Abstract

Investigating validity evidence for standard setting protocols in a pre-clinical medical school curriculum, we tested logistic models with the binary outcome, National Board Success, and various dichotomized cut points including Yes-No Angoff faculty standards, identifying best standards from our examination of predicted probabilities for Success, goodness of fit, and classification accuracy. Odds ratios support interpretations of the consequential aspects of validity evidence.

Our models provided adequate predictive validity evidence for standard setting protocols. Further, classification statistics support arguments for the consequential aspects of our school’s standard setting outcomes. We acknowledge the dynamic nature of testing and thus recognize the imperative for dynamic protocols for standard setting. Accordingly, we advocate for annual reviews of predictive models and classification statistics.

Authors

• Ketsia Dimanche, Mercer University School of Medicine
  Presenting Author

• Edward C Klatt, Mercer University School of Medicine
  Non-Presenting Author

• S. Marshall Angle, Mercer University
  Presenting Author