An Examination of Validity Research on the Mini-Clinical Evaluation Exercise (Mini-CEX): Is the Research Aligned With Contemporary Validity Theory and Does It Meet Established Standards?

• In Event: When Validity Theory Meets Validation Practice: Research Syntheses of Validity Evidence Reported in Seven Areas

Abstract

Objectives or purposes:
This paper provides a systematic review of the validity studies that have been published regarding the Mini-Clinical Evaluation Exercise (“Mini- CEX”, Norcini, Blank, Arnold, & Kimball, 1995) with a view to revealing potential gaps or limitations in the current research. Specifically, we examine the extent to which Mini-CEX studies conform to the major aspects of contemporary validity theory as outlined in the Standards for Educational and Psychological Testing (the “Standards”, AERA, APA & NCME, 1999), specific sources of validity evidence reported for the Mini-CEX, and whether they are aligned with the sources of validity evidence outlined in the Standards.

Theoretical Framework:
The Mini-CEX was developed to assess the clinical skills of internal medicine residents in the US. It is now widely used in many countries to assess medical residents, undergraduate students, practicing doctors and international medical graduates in a variety of specialties and settings, and for both formative and summative purposes. As such, the Mini-CEX carries significant implications for individual students, educational programs, and society. It is critical that decisions arising from the Min-CEX be based on strong validity evidence.

Although there is a body of research that investigates validity evidence for the Mini-CEX, to date no one has examined the extent to which this research meets the recommendations and criteria set out in the Standards or the extent to which all recommended sources of validity evidence are being reported for the Mini-CEX.

Method and Data Sources:
We searched for primary research studies in Academic Search Complete, CINAHL, Education Research Complete, ERIC, MEDLINE, and PsychINFO using search terms “Mini-Clinical Evaluation Exercise” or “Mini-CEX” and “valid*” in all text. We also examined the reference lists from review and summary articles.

We used a multi-step coding process to identify studies and determine: whether they were intended to provide validity evidence; the extent to which the authors’ views of validity conform to the current view of validity; what the authors reported as sources or types of validity evidence; and what other non-validity types of evidence were reported. Finally, we re-coded all evidence presented by the authors as it would be coded according to a framework based on the Standards. We also calculated inter-rater reliability.

Results:
Our results show that the body of Mini-CEX validity research is not well aligned with contemporary validity theory. No study made reference to the Standards. Validity evidence presented by authors would be classified differently when viewed from the perspective of the Standards and some types of validity evidence recommended by the Standards have not yet been investigated. Importantly, there appears to be a lack of theoretical rationale to guide interpretation of scores from the Mini-CEX as would be required by contemporary validity theory.

Scholarly Significance:
This study reveals gaps and limitations in the Mini-CEX validity research. Thus, our results can guide future validation practice. Future research should seek to provide a sound theoretical basis for the Mini-CEX and investigate sources of validity evidence that have not yet been addressed.

Authors

• Dallie A. Sandilands, The University of British Columbia

• Bruno D. Zumbo, The University of British Columbia