Evaluating the Quality of Analytic Ratings With Mokken Scaling

• In Event: 16.026 - Item Response Theory: Advances and Applications

Abstract

Numerous quality-control indicators have been developed to evaluate holistic and analytic ratings based on parametric models, such as the Many-Facet Rasch model (Engelhard, 2013; Wolfe, 2009). Recognizing that parametric model assumptions may be inappropriate for rating data, recent research has explored the use of polytomous versions of Mokken’s (1971) nonparametric scaling models to explore the quality of holistic ratings (Wind & Engelhard, 2014) and rating scales (Wind, 2014) within the context of performance assessments. This study continues the extension of Mokken scaling to performance assessments by exploring the degree to which Mokken-based rating quality indices can be used to explore the quality of ratings assigned within domains on an analytic rubric. Implications for research, theory, and practice are discussed.

Author

• Stefanie Anne Wind, The University of Alabama - Tuscaloosa