A Classroom Observer Like Me: Demographic Congruence Between Teachers and Observers

• In Event: Compensation, Race, and Burnout: Examining Drivers of Disparate Teacher Retention

Abstract

Over the past decade, U.S. states and districts have come to rely on teacher evaluations –and in particular, classroom observations – as a key lever for teacher accountability. Classroom observations are the most frequently used measure of teachers’ on-the-job performance across states and districts implementing post-NCLB teacher evaluation systems, and they typically represent the highest weighted component of a teacher’s summative evaluation rating (Steinberg & Donaldson, 2016). Despite the prevalence of using classroom observations to assess teachers, important questions remain regarding what influences the scoring behavior of the observers, who are usually school administrators (Cohen & Goldhaber, 2016). Recent mixed-methods research based in Los Angeles suggests that administrators draw more often from internal criteria and personal views of teaching – and rely less on observation rubrics – as time passes since their rater-training (Qi et al., 2018). This raises concerns about whether and to what extent teachers’ and administrators’ demographic characteristics influence classroom observation scores.

A growing body of research suggests that administrators’ demographic characteristics and demographic congruence between teachers and principals may play an important role in teacher outcomes and the school context. For example, Grissom and Keiser (2011) find that teachers turn over less and report higher job satisfaction when supervised by a same-race principal. Bartanen and Grissom (2019) find that Black principals decrease Black teacher mobility, as well as increase the likelihood that newly hired teachers are Black.

To further explore the influence of demographic characteristics on teacher outcomes, in this study, I use longitudinal data from a large district in the southeastern United States to examine the relationship between teacher race, the race of the classroom observer, and observation scores. I ask: 1) To what extent are there gaps in teacher observation scores between Black and White teachers? 2) To what extent does demographic matching between teachers and their observers explain Black-White gaps in teacher observation scores?

To address the research questions, I take advantage of the availability of multiple observation scores per teacher per school year and the within teacher-year variation in the demographic characteristics of the classroom observers, who are administrators from the same school. To identify the impact of having a same-race classroom observer, I estimate models that include both teacher-year fixed effects and observer-year fixed effects. The inclusion of teacher-year fixed effects implicitly controls for the characteristics of students assigned to the teacher in that year. The observer-year fixed effects control for the possibility that observers vary in their harshness as raters, as well as the possibility that observers vary in their harshness across years.

Preliminary results indicate that there is a large gap in observation scores between Black and White teachers, with Black teachers scoring lower, even after controlling for experience and school assignment. I also find that the gap in observation ratings between Black and White teachers decreases when Black teachers are rated by Black classroom observers.

Author

• Olivia Chi, Boston University