Accountability and Teacher Evaluation Systems

• In Event: 60.056 - A Role for Quality Improvement in U.S. Educational Accountability?

Abstract

Teacher evaluation systems have become the dominant component of educational accountability efforts, particularly as challenges have been made to the Common Core Standards and assessments. The rationale for these systems has emerged from substantial research on measures of teacher quality, most especially the work of the Measures of Effective Teaching project (Kane, Kerr, & Pianta, 2014). The research has largely focused on issues of validity and reliability of individual measures, particularly those associated with student achievement and with classroom interactions as measured with observation protocols. Additionally, there has been significant attention to combining scores from different measures to arrive at a composite measures (e.g., Mihalyi, McCaffrey, Staiger, & Lockwood, 2013) of teacher effectiveness that are used to make consequential decisions about teachers.

These efforts have revealed the measurement challenges in making valid and reliable decisions about teachers. The current study surveys states’ design and implementation of teacher evaluation systems and, first, describes their fundamental disconnect with general principles of educational measurement findings relevant to measuring teaching quality. I then consider the social, political, performance management, and organizational change factors that can help explain this discrepancy. Finally, I make a set of concrete and tractable recommendations for refining these systems to satisfy basic requirements for an accountability system.

The study is based on an in-depth review of available documentation of teacher evaluation systems supported by the Race to the Top initiative. The information is organized into a conceptual framework that focuses on factors such as system design, specific measure design, quality control processes, local flexibility, reporting, score distributions, statute and policy, and research and validity efforts. Reports for each state are generated, and key administrators from each state are interviewed to confirm, revise, and update any reported findings.

Overall, despite significant variability across factors, a key finding is that teacher evaluation across states cannot be justified as a measurement system. There is no basis to assume that scores reported across districts within states have any common interpretive meaning, either at the individual measure or composite level. There are virtually no quality control measures in place to either ensure or monitor the quality of scores. What is clear is that score distributions for observations are consistently higher than scores produced in research studies.

The unavailability of information to judge evaluation quality, other than basic compliance, makes it almost impossible to characterize and improve the quality of these systems. I close the presentation by discussing a set of steps that states could take to ensure that the systems themselves, and not only teachers, can be the object of study, evaluation, and improvement. Among the most important issues is to consider how consistencies in teacher evaluation results can provide insight into how curriculum, school policy, and other more systemic factors are influencing the quality of teaching, and ultimately, student learning. Current efforts that focus on identifying relatively small proportions of teachers at the tails of performance distributions are likely to miss essential limits in practice that characterize the overall system (Gitomer, 2013).

Author

• Drew H. Gitomer, Rutgers University - New Brunswick/Piscataway