Predicting Family Medicine Specialty Certification Status Using Standardized Measures for a Sample of International Medical Graduates Engaged in a Practice-Ready Assessment Pathway to Provisional Licensure

• In Event: 60.044 - The Best Predictor of the Future May Be the Past, but How Accurate Are the Predictions? Estimating Professional Competence Using Diverse Methodologies

Abstract

Perspective: Canada relies on an important cadre of internationally trained physicians to better serve the health care needs of a growing and culturally diverse population. Currently, one route that IMGs can avail themselves for entry into practice in Canada is referred to as Practice Ready Assessment (PRA). PRA entails entry into some form of - in practice assessment program for IMGs who are deemed to have completed acceptable postgraduate training and/or practice outside of Canada. In 2013, approximately 300 IMG physicians were assessed through seven existing PRA programs in family medicine and/or other specialties.

Objective: The purpose of the present study was to assess the usefulness of MCC examination scores as well as a number of educational/socio-demographic variables in predicting pass/fail standing on each component of a Family Medicine certification examination as well as whether time or number of attempts to pass, for a sample of successful PRA candidates.

Methods and Data Sources: Our data set included a sample of 132 candidates who had successfully completed a PRA program between 2007 and 2011 and attempted the certification examination in Family Medicine by the end of 2012. Predictors included a number of socio-demographic predictors, the most recent MCCEE, MCCQE Part I and II scores as well as the number of attempts required to pass both components of the family medicine certification examination. The main outcome was the pass/fail status on each certification exam component. Logistic regression modeling was used to estimate (discrete) pass/fail status as a function of our predictors while discrete survival analysis was employed to additionally model number of attempts as a predictor. This study was approved by the Ottawa Hospital Research Ethics Board.

Results: The results of the logit-linear modeling are shown below.
Simulated Office Orals (SOOs) Short-answer Management Problems (SAMPs)
Predictors Β Χ2-value Type I Error β Χ2-value Type I Error
Intercept 5.26 2.40 0.12 -4.94 7.68 0.01
Gender -2.61 11.45 0.01 -1.02 5.86 0.02
MCCQE Part I Score 0.00 0.78 0.38 0.02 11.88 0.01
Number of Attempts
(2 vs. 1) 1.19 4.18 0.04 -0.41 0.91 0.34
Model Fit L2 (7)=37.90; p>0.01 L2 (4)=37.55; p<0.01

With respect to the SOOs component, both gender and the number of attempts significantly predicted pass/fail status, L2(7)=37.90, p<0.0001. With regard to the SAMPs component, only gender and the most recent MCCQE Part I score significantly predicted pass/fail status; L2(4)=37.55, p<0.0001. Unlike the SOOs component, repeating the SAMPs does not significantly increasing the odds of passing.

Significance: The present study was aimed at providing evidence to support and inform a critical phase of the PRA process to all programs that is screening, in the hopes of better informing a pan-Canadian strategy that will ultimately be beneficial to IMGs, MRAs and the Canadian public in the most cost-effective manner. The implications of the findings reported in this study will be discussed in light of this coordinated national strategy for PRA.

Authors

• Andre F. De Champlain, Medical Council of Canada

• Cindy Streefkerk, Medical Council of Canada

• Fang Tian, Medical Council of Canada

• Marguerite Roy, Medical Council of Canada

• Sirius Qin, Medical Council of Canada

• Carlos A. Brailovsky, Université Laval

• Paul Rainsberry, College of Family Physicians of Canada

• Tim Allen, College of Family Physicians of Canada