Understanding the Impact of IRT (Item Response Theory) Item Parameters and Latent Distribution Shape on the Reliability of Total Scores

• In Event: Item Response Theory: Applications and Extensions

Abstract

Previous research on the optimal number of scale categories is inconsistent. Some researchers found evidence that reliability does not improve beyond four categories whereas others suggested the need for as many as nine categories. This paper presents equations based upon a polytomous IRT model for computing the reliability of total scores as a function of item locations, category thresholds, and item discriminations. An important innovation is the use Fleishman’s probability distribution to show that nonnormality in the latent trait distribution reduces the reliability of total scores. The results imply that, in addition to other factors, nonnormality of the latent distribution and dependency between item discriminations and number of scale categories impact total score reliability.

Author

• Steven Andrew Culpepper, University of Illinois at Urbana-Champaign