Examinee Classification Decisions in the Presence of Residual Dependencies: A Simulation Study

• In Event: Poster Session 4
  In Poster Session: Division D Section 1 Poster Session

Abstract

Previous studies have identified negative impacts of ignoring violations of local item independence in Item Response Theory (IRT) models, including inflated reliability along with biased item parameter and ability estimates. However, it is not known whether ability biases are severe enough to impact cut-point classification decisions. This simulation study examines whether the strength of secondary traits, the size of testlets, or the distribution of ability impacts sensitivity and specificity of classifications when utilizing three different models: unidimensional IRT, Testlet Response Theory, and bi-factor approaches. Results confirm that the bi-factor model displays superior sensitivity and specificity in classifications when local item independence is violated, and that application of unidimensional models in the face of residual dependencies impacts decisions made about individuals.

Authors

• Jonathan D. Rubright, American Institute of Certified Public Accountants

• Ratna Nandakumar, University of Delaware