Fully Bayesian Model Selection Methods in Diagnostic Classification Models

• In Event: Hilton Roundtable Session 6
  In Roundtable Session: Division D Section 1 Roundtable Topic: Cognitive Diagnostic Models

Abstract

In the Bayesian literature, the leave-one-out cross-validation (LOO) and the widely available information criterion (WAIC) are two popular model selection approaches taking advantages of the posterior distribution, instead of relying on point estimates. Although their applications have been seen in other psychometric modeling frameworks, how their advantages translate into diagnostic classification models remains unknown. In this paper, a comprehensive simulation study was conducted to compare the performance of LOO and WAIC judged by three fit indices—AIC, BIC, and CAIC. The simulation outcomes show that LOO and WAIC perform well in many situations, especially when the data generated come from complex models. The findings also caution users to properly choose priors as they may affect the results.

Authors

• Zhehan Jiang, Peking University

• Wenchao Ma, The University of Alabama - Tuscaloosa

• Randall E. Schumacker, The University of Alabama

• Jihong Zhang, The University of Iowa