Contributing to Guidelines on the Use of Competency Evaluations in Health Professions Education

• In Event: Evaluation of Competence in Competency-Based Education

Abstract

In the health professions, competency evaluations have surfaced as a potentially useful assessment method. This paper examines validity evidence for competency evaluations that can contribute to developing guidelines. Data for 2,701 end-of-rotation evaluations measuring milestones for 142 residents were analyzed. Generalizability analysis and higher-order confirmatory factor analysis were used to examine the internal structure of ratings. Psychometric implications for combining evaluations scores using composite score reliability were examined. Guidelines for rotation evaluations proposed in this study provide useful solutions that can help Program Directors make decisions on resident progress and contribute to assessment systems in graduate medical education. Broad implications for translating findings to professions education are discussed.

Authors

• YoonSoo Park, University of Illinois at Chicago

• Fred Zar, University of Illinois at Chicago

• John J. Norcini, Foundation for Advancement of International Medical Education and Research

• Ara Tekian, University of Illinois at Chicago