Using Simulation to Evaluate Classification Consistency

• In Event: Methodological Considerations for Rasch Measurement Research

Abstract

Procedures and software applications exist to estimate classification consistency (CC) with a single test administration. However, practical guidance to support interpretation and evaluation remains relatively absent (Subkoviak,1988). In this study, we examine whether simulated data may help practitioners interpret and evaluate CC. We estimate CC with simulated data, generated with near-perfect model-data fit, and compare it to the CC estimates from operational data. As expected, the CC estimates for the simulated data were higher than those for the empirical data. We interpreted the differences as the impacts of minimal levels of model-data misfit on misclassification. Perfect model-data fit is never observed in practice, but simulated outcomes provide an ideal to strive toward and where to focus improvement efforts.

Authors

• Angela Adrienne Walker, Ascend Learning

• Kari Hodge, Ascend Learning