What Knowledge, Skills, and Attitudes Do Physicians Judge to Be Important for Entry Into Supervised and Unsupervised Practice?

• In Event: Developing a Blueprint for a National Medical Licensing Examination Program: From Soup to Nuts

Abstract

Objective

The purpose of this survey was to collect physician judged importance of knowledge, skills and attitudes (KSAs) required at entry into supervised practice (residency) as well as unsupervised practice (initial licensure).

Perspective

For licensure examinations, a practice analysis is essential as it helps to determine what domains should be measured in any given exam form as reflective of “reality” (i.e. actual practice

A survey to collect physicians’ importance judgments of various medical tasks was carried out to inform the blueprint and associated test specifications for a national medical licensing examination.

Methods

Our survey included: (1) medical expert (e.g., “anxiety”), and; (2) Non-medical expert (e.g., “appropriately develop and maintain ethical relationships”) KSA questions, as defined by the CanMEDS framework (Frank, 2005). Most of the medical expert questions were also presented in the context of determining cause and initiating management. A four-point Likert scale was used to collect importance judgments.

Results

For summary data analytic purposes, the four-point scale categories (“Not Important”, “Moderately Important”, “Very Important”, “Extremely Important”) were collapsed to “Not/Moderately Important” and “Very/Extremely Important”.

The 487 KSA questions were repeated for the unsupervised and supervised decision points. A total of 1,836 physician respondents completed the survey resulting in an overall response rate of about 10%.

122 out of 487 questions (25.1%) were judged to be Very/Extremely important by 80% or more of the respondents at the supervised level.

Approximately 25% of these related to determining cause whereas 20% and 3% respectively, were associated with initiating management and differentiating between categories.

With respect to CanMEDS roles, approximately
22% of questions judged to be Very/Extremely important by 80% or more of the respondents at the supervised level fell in the Professional category, while 16%, 10%, 2%, 2% respectively corresponded to the Communicator, Collaborator, Scholar, and Manager roles.

327 out of 487 questions (67.1%) were judged to be Very/Extremely important by 80% or more of the respondents at the unsupervised level.

Approximately 31% of these 327 questions related to determining cause, while 36% and 3% respectively, referred to initiating management and differentiating between categories.

Approximately 11% of the latter 327 questions corresponded to the Professional CanMEDS role while 7%, 6%, 2%, 2%, 1% respectively were related to the Communicator, Collaborator, Scholar, Manager, and Health Advocate roles.

There was complete overlap in survey questions that were deemed “Very/Extremely Important” across the supervised and unsupervised levels. Non-medical expert questions were proportionally as important as the medical expert questions suggesting that items relating to the Collaborator, Communicator and Professionalism roles are important to assess at the supervised level. For the medical expert questions, slightly more emphasis was placed on questions that pertained to determining cause at the supervised level whereas more prominence was given to questions which related to initiating management at the unsupervised level.

Scholarly significance

The results of this survey were critical in aiding subject matter experts to develop a defensible blueprint and accompanying test specifications for a national medical licensing examination program that reflect expectations of practicing physicians across the country.

Author

• Andrea Gotzmann, Medical Council of Canada