A Systematic Review of Validity Evidence for ACGME (Accreditation Council for Graduate Medical Education) Milestones in Surgical Specialties

• In Event: Assessment Innovation in Professions Education

Abstract

This review systematically synthesized studies that reported validity evidence for Accreditation Council for Graduate Medical Education Milestones in surgical specialties. A total of 114 studies were included. The primary source of validity evidence (39%) was relations to other variables (knowledge and skills, patient/health care, social-emotional variables), followed by response processes (33%: inter-rater reliability, rating processes, structure of CCC, rater training, longitudinal reliability, straight-lining) and consequences (25%: intended use, anticipated impact, value and utility). Only 12 studies (11%) reported internal structure evidence (variability and discrimination, internal consistency reliability, factor analysis). This review provides an operational definition of validity evidence for Milestones and highlights areas needing further research to support the use of Milestones in a CBME system.

Authors

• Ting Sun, University of Utah

• Stan Hamstra, University of Toronto

• Kenji Yamazaki, Accreditation Council for Graduate Medical Education

• Katherine Jiawen Ren, University of North Carolina - Charlotte

• Brigitte K. Smith, University of Utah