Are Teacher Candidate Ratings Reliable? What Many-Facet Rasch Measurement Says About Preservice Teacher Supervisor Ratings

• In Event: Classroom Observation SIG Roundtable

Abstract

A central component of teacher preparation efforts is the observation and rating of preservice teachers by their supervisors. However, the reliability of preservice teacher ratings is not well documented. Using a revised version of the Model of Appropriate Teaching for Teacher Candidates rubric, we apply Many-facet Rasch measurement to evaluate the ratings of ten preservice teacher supervisors for eight prerecorded lessons of teacher candidates following calibration training. Results suggest that supervisors provided reliable ratings across varying levels of teacher candidate ability (Rel = .95). Results also suggest that supervisors were mostly self-consistent across ratings and used the rubric in its intended manner. However, supervisors differed noticeably in their level of severity, and not all supervisors achieved acceptable levels of self-consistency.

Authors

• Eli Andrew Jones, The University of Memphis

• Stefanie A. Wind, The University of Alabama - Tuscaloosa

• Anna Hart, Columbus State University

• Jan Burcham, Columbus State University

• Thomas Dailey, Columbus State University