How Do Administrators' Uses of an Observation Protocol Change Over Time?

• In Event: Classroom Observation: Instruments, Application, and Findings

Abstract

To understand the validity of ratings in practice and how administrators’ use of the observation tool evolve, this study investigates how the cognitive aspects of administrators’ use of observations change as they practice. Results show that despite extensive training and certification, administrators still used a wide variety of reasoning strategies. Findings revealed the trend that administrators relied less on the scoring protocol, used more internal criteria, and more often approached scoring from the perspective of helping teachers improve. If observation ratings are used to support high stakes decisions, administrators should be trained to reliably use reasoning strategies and scoring criteria that reflect both the rubrics and the specified rating procedures. Implications for improving the training and evaluation systems are discussed.

Authors

• Yi Qi, Educational Testing Service

• Courtney A. Bell, ETS

• Margaret Wilson, Educational Testing Service

• Amanda Redash, Boston University

• Nathan Jones, Boston University

• Jennifer M. Lewis, Wayne State University