Redesigning Feedback Reports for Program Directors

• In Event: 75.016 - Symposium on Recent Developments in Medical Certification Score Reporting

Abstract

Program directors are responsible for a variety of tasks including program management (i.e., accreditation, program evaluation), and supervising the medical education and training of fellows. Accordingly, program directors often want feedback about how their candidates performed on initial certification exams. The American Board of Internal Medicine (ABIM) has been providing feedback to program directors for many years. The program director reports, like the individual score reports, were originally printed and paper copies were mailed to program directors. This approach was time-intensive and made tracking data across years challenging for program directors.
Recently, ABIM updated the individual score reports provided to candidates in an effort to deliver more meaningful feedback in an electronic format. This initiative led ABIM to revisit the feedback we provide to program directors. Our purpose for the program director report redesign was twofold. First and foremost, we wanted to design a report with improved content that more closely aligned with the individual score reports and better met the needs of our program directors. In addition to ensuring useful content, we also aimed to create a platform that would allow us to more efficiently generate program director reports in an electronic format that could be viewed and saved by program directors.
In order to meet program director needs, we used a variety of approaches to collect feedback about what information would be most helpful in a program director report. To better understand program director preferences, we sent out a survey with several prospective figures and tables to all Internal Medicine program directors and coordinators. We also attended a national conference and conducted phone interviews throughout the development process to gather program director opinions and prioritize the most pertinent needs. In addition to content suggestions, one of the highest priorities for program directors was to receive information more quickly than in the past. Many program directors specifically hoped to have information about the pass/fail status of their candidates closer to when the individual candidates find out.
Our enhanced reports are the result of multiple rounds of feedback from program directors. The tables and figures align well with the individual score reports allowing for a more cohesive approach to providing feedback. The reports were incorporated into an evaluation system already used by all program directors of ABIM subspecialties. This will allow the program director to access electronic versions of their reports for several years of data. In addition, we altered our process such that the program directors now receive two reports. First, they receive the pass/fail status of their candidates immediately after results are released to individual candidates. Second, they receive the full report with summary data once all data has been collated. Ultimately, the collaboration with program directors helped to create a more effective and meaningful report.

Authors

• Kelli Samonte, American Board of Internal Medicine

• Bradley G. Brossman, American Board of Internal Medicine