A Higher-Order Cognitive Diagnostic Model for Large-Scale, Spiral Assessment Designs

• In Event: Advancing Diagnostic Psychometric Models to Enhance Student Learning: Statistical Theory, Methods, and Applications

Abstract

In large-scale assessments, such as the Programme for International Student Assessment (PISA), students are only assigned a subset of the total items to minimize the burden on students and schools. There is increasing interest on the utility of diagnostic methods for providing policymakers with a fine-grained understanding of population skill profile distribution. Specifically, knowledge about the distribution of skills in the student population could provide researchers and decision-makers with insights about skill strengths and weaknesses in an internationally normed assessment. The purpose of this study is to present new methods for inferring the latent structure of cognitive diagnostic models for large-scale assessments that attempt to offer a broad measure of achievement. There has been considerable attention on response data being fully present among data collected. Within this study, we propose a novel Bayesian higher-order formulation for handling missing data on large-scale assessments within a cognitive diagnosis framework that follows from a spiral assessment design. We report the performance of the model by conducting Monte Carlo simulations and applying the method on real-world data obtained from the 2012 PISA mathematics exam.

Authors

• Steven Andrew Culpepper, University of Illinois at Urbana-Champaign
  Non-Presenting Author

• James J. Balamuta
  Presenting Author