Aggregating Workplace-Based Assessment and Entrustable Professional Activity Ratings using Bayesian Inference

• In Event: Pushing Measurement Forward: Application of Advanced Quantitative Methods

Abstract

Smartphone-based workplace-based assessment and entrustable professional activity systems have led to an increased volume of data for training programs. Program leadership and faculty need strategies for synthesizing multiple micro-assessments into valid, fair, and reliable educational decisions. This paper describes an explainable, probabilistic modeling approach that aggregates micro-assessments using Bayesian inference networks (BNs). Using a set of calibrated micro-assessments from the Society for Improving Medical Professional Learning’s (SIMPL) general surgery app, SIMPL OR, this paper provides an overview for how BNs can generate interpretable overall operative performance scores. Next steps based on this approach include using BNs to set performance standards and connecting to competency models like the Accreditation Council of Graduate Medical Education’s Milestones.

Author

• Andrew E. Krumm, University of Michigan