Certification Exam Passing Score: Angoff and Receiver Operating Characteristic

• In Event: Marriott Roundtable Session 33
  In Roundtable Session: Applied Psychometric Methods in Validation Research

Abstract

In the credentialing field, the Angoff method is most widely used for standard setting. Recent work on methods for evaluating standard setting have called into question the robustness of the Angoff method and suggest that other methods should be used concurrently with the Angoff method to provide evidence of the standards set. One such method is receiver operating characteristic (ROC) curve analysis. Using data from a credentialing program in the corrosion industry, we used ROC analysis to evaluate the optimal differentiation between passing and failing candidates. Results showed that the ROC analysis (area under the curve, AUC = .69) recovered the same passing threshold as standard setters. Implications for concurrent use of the Angoff method and ROC analysis are discussed.

Authors

• Kari Hodge, NACE International Institute

• R. Noah Padgett, Baylor University