A Test for Testlet Effects in Cognitive Diagnostic Models

• In Event: Roundtable Session 28
  In Roundtable Session: Issues and Practical Applications in Cognitive Diagnostic Modeling

Abstract

Cognitive diagnostic models (CDMs) are used to diagnose the presence or absence of a set of skills. The validity of a CDM depends on whether item responses are a set of independent random variables conditional on the latent variables of a correctly specified model. Testlets, a cluster of test items that shares a common stimulus, call into question the assumption of conditional independence because the items measure something in common other than the attributes the model specifies. This study proposes a Mantel-Haenszel (M-H) test to detect testlet effects at the pairwise item level. The proposed method is thoroughly studied in simulation under a wide variety of conditions, and is illustrated with the PTE Academic reading proficiency assessment data.

Authors

• Youn Seon Lim, University of Illinois at Urbana-Champaign

• Jeffrey Douglas, University of Illinois at Urbana-Champaign