Examining Raters Effects Using the Many-Facet Rasch Model in Mathematics Teacher Classroom Performance Assessment

• In Event: Rasch Modeling in Educational Settings

Abstract

Raters are an important source of variability in examinee scores in rater-mediated assessment. In this study, we investigated effects related to raters via many-facet modeling in the assessment of 47 elementary school mathematics teachers’ classroom teaching performance by 16 raters. We examined five types of rater effects, including rater severity, central tendency, halo, randomness, and differential severity effects. The results indicated a small amount of variability in rater severity and no evidence of central tendency, halo, or differential severity effects towards teachers who taught different grades of students. Two raters, however, exhibited some randomness in their ratings and two raters showed differential severity effects towards the two groups of teachers.

Authors

• lanrong li, Amplify Education, Inc.

• Robert Schoen, Florida State University

• Insu Paek, Florida State University