The Accuracy of Identifying Attribute Hierarchies in Classroom Diagnostic Assessment

• In Event: Virtual Poster Session
  In Virtual iPresentation Only: Division D - Section 1: Educational Measurement, Psychometrics, and Assessment Virtual Poster Session

Abstract

This simulation study aims to investigate the accuracy of identifying hierarchical relationships among attributes with the maximum likelihood estimation (MLE) in the context of classroom formative assessments. The fit indices, AIC and BIC, were used to choose the best-fitting hierarchical model. The manipulated variables included hierarchical structure, number of items and attributes, sample size, and item quality. The r package, CDM, was used to generate simulated datasets and estimate CDM models. The Q-matrix was a simple structure with one item measuring one attribute. The results found that with 3 attributes, BIC performed much better than AIC, whereas with 5 attributes, AIC and BIC performed equivalently well. Item quality had a high impact on the selection of accurate hierarchical models.

Authors

• Yi-Hsin Chen, University of South Florida

• Chia-Ling Hsu, Hong Kong Examinations and Assessment Authority

• Yi-jhen Wu, TU Dortmund University

• Xin Qiao, University of South Florida - Tampa