Generalizability of Clinical Performance Assessment Scores Across and Within Clerkships at a New Medical School

• In Event: 64.044 - Measuring Learner Differences and Variability

Abstract

This study examined the reliability of a clinical performance assessment form and determined the number of faculty raters needed to create a reliable estimate of student performance in the core clerkships at a new medical school. Results of a generalizability study (G study) revealed a range of G-coefficients of .10-.78, indicating extremely low reliability in one clerkship and acceptable or near-acceptable reliability in the others. Decision analysis (D study) results revealed 4 to 9 raters are needed to achieve acceptable reliability for clerkships in the near-acceptable or acceptable range. Results have implications for faculty assessment training efforts and guidelines for number of raters per student in the clerkships to improve use of the assessment form and reliability.

Authors

• Stephanie B. Corliss, The University of Texas at Austin

• Yoon Soo Park, University of Illinois at Chicago

• LuAnn Wilkerson, The University of Texas at Austin