Estimating Interrater Reliability in Ill-Structured Measurement Design: An Example From Charter School Fund Applications

• In Event: Poster Session 4
  In Poster Session: Division D Section 1 Poster Session

Abstract

Inter-rater reliability is an important measure of the quality of rating scores. The inter-rater reliability estimates in ill-structured measurement designs (ISMDs) are different from those in fully crossed or nested designs. The proposed study investigates the effectiveness of newly developed inter-rater reliability G(q,k) in ISMD using data obtained from raters’ scores on charter school fund applications.

Authors

• M. David Miller, University of Florida

• Wei Xu, University of Florida

• Nancy Thornqvist, University of Florida