COVID-19's Impact on Family Medicine Residents' Exam Performance Trajectory

• In Event: Loss or Gain? Evaluating the Impact of the COVID-19 Pandemic Across the Medical Education Continuum

Abstract

Purpose
An unprecedented two-year-long COVID-19 pandemic fundamentally disrupted the normalcy of family residency training (Awadallah et al., 2021; Çevik & Ungan, 2021). Clinical activity decreased 81% during the first pandemic surge, resulting in a substantial loss of patient care interactions for residents (Awadallah et al., 2021). Traditional face-to-face learning was partially replaced with distance learning across most of the medical education periods. In this study, we aim to quantify the impact of family residency training disruption using exams administered by the American Board of Family Medicine (ABFM).

Methods
The participants in the study were family residency residents who took the In-Training Examination (ITE) from 2011 to 2021 as postgraduate year 1 (PGY-1), PGY-2, PYG-3 residents, or the Family Medicine Certification Examination (FMCE). During this study period, ITE and FMCE scores were equated onto a common scale (ranging from 200 to 800), permitting direct growth comparisons. This study focuses on three score growth sequences from 2011 to 2021: PGY-1 to PGY-2, PGY-2 to PGY-3, and PGY-3 to FMCE.
Our primary hypothesis is that all score growth in 2020 and 2021 would be smaller than in previous years, possibly due to the disruptions of COVID-19. Secondly, we hypothesize the score growth deficiency during 2020 and 2021 would be more salient for residents in the early stages of training compared with the final stages (i.e., growth from PGY-3 to FMCE), due to devoting more exam preparation efforts to FMCE (high-stakes) than ITE (low-stakes).
To ensure the comparability of training experience, residents with irregular training progression were excluded from this study (i.e., repeating a training level or dropping out of the training program), resulting in 37,492, 35,996 and 33,431 residents in each score growth period analysis, respectively.

Results
Figure 1-3 present results for three growth periods. Within each figure, the x-axis denotes the calendar year, and y-axis denotes the scaled score growth. The red dashed line indicates the mean score growth across these examination periods. It can be observed that the magnitude of score growth in 2020 and 2021 is noticeably below the red dashed line. To formally test this observation, we compared the score growth between the “before COVID” group (before 2020) and the COVID group (2020 and 2021) via a series of t-tests for these three score growth periods. Table 1 shows the group means, standard deviation, along with group mean difference, and 95% confidence interval of the group mean difference. During the PGY-1 to PGY-2 and PGY-2 to PGY-3 growth periods, the growth differences between the “before COVID” group and COVID group are substantive, 34.4 and 34.0 in the scaled score, whereas the growth difference between these groups during PGY-3 to FMCE is small (4.5; 95% CI [2.2, 6.7]; p <0.001).

Significance
The results supported our hypothesis that score growth in pandemic years is significantly smaller compared to previous years. The decreasing magnitude of score growth is more salient in early stages such as PGY-1 to PGY-2 and PGY-2 to PGY-3 period compared with the PGY-3 to FMCE stage.

Authors

• Ting Wang, American Board of Family Medicine
  Presenting Author

• Thomas R. O'Neill, American Board of Family Medicine
  Presenting Author