Comparison of Attribute Coding Procedures for Retrofitting Cognitive Diagnostic Models

• In Event: Poster Session 9
  In Poster Session: Applications of Cognitive Diagnostic Models: From Q-Matrix Development to Data-Model Fit

Abstract

This paper explores the process of coding items when retrofitting a cognitive diagnostic model to determine if the coding process impacts model-data fit. Three approaches for coding items for the cognitive attributes required to provide a correct response were compared. The coding approaches were implemented with three groups of coders: a group without additional training, a group receiving a training set, and a group of content experts reaching consensus. Inter-rater reliability was calculated for the first two groups, and the resulting Q matrices for the groups were compared to determine which approach achieved the best fit to the data using the Ox Metric software.

Authors

• Amy Clark, The University of Kansas

• Neal M. Kingston, The University of Kansas