In-Training Examinations: All About Predictive Validity and Meaningful Feedback

• In Event: On the Use and Validity of Tests in the Professions

Abstract

Many professional certification programs offer in-training examinations to trainees as a self-assessment instrument prior to actual certifying examination. It is critical to provide predictive validity evidence and meaningful feedback to test users for proper interpretation and usage of the test results. This study evaluates the predictive validity of a medical subspecialty in-training exam (SITE). Linear regression is used to analyze predictive nature at each of the three training levels. Logistic regression is used to evaluate the overall predictive accuracy of the SITE on the certifying examination pass/fail classification. Sample score reports provided to fellows and training programs are shared to show the effort of trying to provide meaningful feedbacks.

Authors

• Ying Du, American Board of Pediatrics

• Robert Thomas Furter, American Board of Pediatrics

• Linda A. Althouse, American Board of Pediatrics

• Rachael Jin Bee Tan, Schroeder Measurement Technologies, Inc.