Cross-Classified Measurement Modeling of Teacher Observation Ratings From Multiple Raters

• In Event: 66.052 - Estimation Accuracy and Conditions for Complex Multilevel Models

Abstract

This study demonstrates the use of a cross-classified random effects model (CCrem) to estimate the impact of classroom characteristics on teacher observation ratings when observed by multiple raters. The CCrem multilevel measurement model (MMM), which accounts for both measurement error and rater bias, was compared to the conventional MMM, which does not account for rater bias. Drawing on real data from the Measures of Effective Teaching (MET) study, we found that the intercept estimates were unaffected, the standard error estimates of the intercept were lower under the MMM, the rater bias variance was reallocated to the measurement error variance under the MMM, the CCrem MMM fit better to the data, and teacher effectiveness rankings differed between the two estimating models.

Authors

• Audrey J. Leroux, Georgia State University

• Kevin Fortner, Georgia State University

• Hongli Li, Georgia State University