Optimal Number of Procedures for the Assessment of Intraoperative Autonomy and Performance of Surgical Trainees

• In Event: Validity Evidence Across Professions Education Settings

Abstract

The System for Improving and Measuring Procedure Learning (SIMPL), a smart phone based operative assessment application, was developed to assess surgical residents’ intraoperative performance. It is critical to establish consistent and proper use of the SIMPL tool across varying procedure conditions. This study adopted a multivariate generalizability theory to estimate the error variance attributable to procedures and determine the optimal number of procedures for a reliable assessment. Results revealed that variance components for procedures are consistently small and smaller variance components were associated with attending faculty compared to trainees. Additionally, the optimal number of procedures required for a reproducible assessment was 10, 17, 15, and 17 for Postgraduate Year (PGY) 2, PGY 3, PGY 4, and PGY 5, respectively.

Authors

• Ting Sun, University of Utah

• Stella Yun Kim, University of North Carolina - Charlotte

• Brigitte K. Smith, University of Utah