Cross-Classified Model of Repeat Examinee’s Ratings of Humanistic Competence an a Medical Licensure Examination Assessing Clinical Performance

• In Event: Measuring the Noncognitive Traits of Students in the Professions

Abstract

Examinees were observed with improved performance on subsequent attempts on licensure examinations. Benefit of previous exposure to material on performance is equivocal. This study investigated standardized patient (SP) gender effects and pre-exposed material on repeat examinees’ ratings of humanistic competence on a medical licensure performance assessment. Using cross-classified multilevel model methods, examinee, SP and portrayed case were treated as random effects, providing means to assess unwanted variation attributed to SPs and cases on ratings. Inconsistent results of SP gender and repeat case effects suggest neglect of specifying the complex multilevel structure could lead to imprecise estimates of coefficients and SEs in the analysis resulting in incorrect extrapolation. Implications address the administration of prior exposure of material in a SP-examination.

Authors

• Xiaoshu Zhu, University of Maryland - College Park

• William L. Roberts, National Board of Osteopathic Medical Examiners

• Xiuyuan Zhang, The College Board