The Effects of Q-Matrix Misspecification on Classification Accuracy and Consistency for Cognitive Diagnostic Assessment

• In Event: 18.020 - Research on Cognitive Diagnostic Modeling

Abstract

The present study investigates the effects of Q-matrix misspecification on diagnostic classification accuracy and consistency of cognitive diagnostic assessment. The two types of Q-matrix misspecifications examined are Q-entry misspecification (which includes three levels of misspecification: 10%, 20%, 30%) and attribute misspecification (which includes attribute exclusion and attribute inclusion). Results of a simulation study show that both Q-entry misspecification and attribute misspecification significantly deteriorate classification accuracy and consistency of diagnostic results. In addition, the classification accuracy and consistency indices have the potential to be useful in identifying possible attributes misspecification (e.g., attribute inclusion) in empirical analyses. These results underscore the importance of correct Q-matrix specification in the development of cognitive diagnostic assessments.

Authors

• Cong Chen, University of Illinois at Urbana - Champaign

• Jinming Zhang, University of Illinois at Urbana-Champaign

• Shenghai Dai, Washington State University - Pullman