Establishing a Quality Control for Residency Applicant Scores

• In Event: 34.049 - Selection and Prediction

Abstract

The residency program selection process incorporates application review and candidate interviews to create an ordered rank list. Though this is an important process for determining the department’s future trainees, the system lacks a quality control mechanism by which the faculty ratings for ranking are scrutinized. This study used many-facet Rasch measurement (MFRM) to establish a quality control system for the candidate selection process. A total of 1378 observations from 158 candidates were used in the MFRM model, explaining 58.42% of the variance in the data. Fit indices indicated that one of the five judges inconsistently applied the rating scale. MFRM results showed some unexpected observations based on standardized residuals that helped focus on specific instances for diagnosing measurement error.

Authors

• Jed Wolpaw, Johns Hopkins University

• Gillian Isaac, Johns Hopkins University

• Tina Tran, Johns Hopkins University

• Michael C Banks, Johns Hopkins University – Baltimore

• Steven Beaudry, Johns Hopkins University

• Priyanka Dwivedi, Johns Hopkins University

• Serkan Toy, Johns Hopkins University