Development of an Objective Structured Clinical Examination for Preparation for the American Board of Anesthesiology APPLIED Certification Examination

• In Event: Frontiers in Reporting Test Performance—Paper Session

Abstract

In March 2018, the American Board of Anesthesiology will implement an OSCE component into the final part of anesthesiology residents’ applied exam.
This study describes the validity and reliability of an OSCE assessment of anesthesiology residents at a large residency program.
Fourteen CA-3 anesthesiology residents were assessed on nine stations by two case-specific raters. The OSCE assessment achieved overall 98% agreement, indicating excellent inter-rater reliability. In addition, using generalizability theory (G theory) analysis, this study yielded a phi-coefficient of 0.67 and a G-coefficient of 0.76, suggesting that most of the study’s variance can be attributed to items nested in station interaction.
Overall, this study demonstrates the feasibility and utility of an OSCE assessment in preparing CA-3 residents for high-stakes exams.

Authors

• Pedro Tanaka, Stanford University

• Aileen Adriano

• Yoon Soo Park, University of Illinois at Chicago

• Alex Macario