Evaluating Model Fit in Bayesian IRT With Small Samples

• In Event: Modern Psychometric Methodologies and Applications

Abstract

The application of Bayesian methods in assessing the reliability and validity of educational and psychological instruments, such as Bayesian CFA and Bayesian IRT, is growing. However, research on model fit statistics for model diagnostics within Bayesian CFA or IRT is limited. This study introduces two novel statistics for assessing model-data fit within Bayesian IRT: posterior predictive model check (PPMC) and a Bayesian variant of the root mean square error of approximation (BRMSEA). A simulation study compares the performance of these methods, considering model misspecification, sample sizes, and prior information. The 90% posterior probability interval of the BRMSEA, with cutoff values as .1 for the upper limit, is found valid for evaluating model fit, even with small sample sizes.

Authors

• Jihang Chen, Boston College

• Zhushan Mandy Li, Boston College