Strengthening Rating Scale Functioning: A Rasch-Informed Measurement Evaluation Framework for Sociology

• In Event: Student Forum Roundtables Session
  In Refereed Roundtable (90 minute): Roundtable 20: Sociological Theory and Methods

Abstract

In recent decades, sociologists have increasingly drawn on psychometric frameworks (e.g., Classical Test Theory and Item Response Theory) to develop and evaluate instruments so they behave in ways theoretically expected. Additionally, statistical procedures are utilized to generate evidence for reliability and validity. However, statistical procedures alone do not directly evaluate rating scale functioning. To support statistical procedures, the Rasch model, developed by Georg Rasch in 1960, is a probabilistic measurement theory that offers explicit criteria for evaluating rating scale functioning. This paper illustrates a case study where a Rasch analysis was performed on a six-item forgiveness scale exhibiting structural misalignment (i.e., nonmonotonic category functioning, disordered thresholds, and reverse scale directionality). The case demonstrates how Rasch assumptions and expectations within measurement and instrumentation can strengthen rating scale functioning through structural refinements. Moreover, the case study exemplifies how instruments can demonstrate strong validity and reliability statistics, and still have suboptimal rating scale functioning. The findings from the case study were used to develop a Rasch-informed measurement evaluation framework to illustrate how Rasch assumptions (i.e., unidimensionality, invariant measurement, scale directionality, ordered thresholds, and monotonic category functioning) can be applied to existing and developing instruments to improve rating scale functioning in sociological instrumentation.

Author

• Kenya Johnson, Morgan State University