Measuring Teaching Quality Using Student Achievement Tests: Lessons From Educators’ Responses to No Child Left Behind

• In Event: Assessing Teacher Quality: Understanding Teacher Effects on Instruction and Achievement

Abstract

Objectives
States and districts across the U.S. are adopting policies requiring teacher evaluations to be based in part on student achievement growth. The idea that teachers should be held responsible for promoting student learning, and that learning can be measured adequately through standardized tests, has common-sense appeal. However, efforts to implement these policies have generally not addressed the implications of using tests for this purpose. This paper explores the likely consequences of test-based teacher evaluation and offers recommendations for developing systems that will promote improved teaching and learning.
Perspectives
Responsible test use requires investigating the validity of scores on those tests for the intended purposes (AERA, APA, & NCME, 1999; Kane, 2006). Advocates of test-based teacher evaluation often claim that it will improve student learning by motivating teachers, giving teachers useful data for adjusting instruction, and providing decision makers information to use in placing, supporting, and rewarding teachers (see, e.g., chapters in Goldhaber & Hannaway, 2010). It is critical for those who mandate this use of tests and those who implement these systems to consider what we know about the kinds of decisions that these systems can support and the consequences they might promote.
Methods and Data
The paper draws on a study of educators’ responses to test-based accountability in three states and on literature on teacher evaluation and compensation. The study gathered the perspectives of teachers and administrators at all levels of the system, from the state to the classroom, using surveys, interviews, and analysis of administrative data over three years. Survey constructs included self-reported changes in practice, engagement in professional development, support for aspects of accountability reforms, and perceived effects on students and school climate. We selected representative samples of districts in each state, and representative samples of elementary and middle schools in each district. All principals and all teachers who taught mathematics or science in grades 3, 4, 5, 7, or 8 were surveyed, and a subset of principals, teachers, and parents were interviewed.
Results
The paper draws on these data and other sources to identify implications for test-based teacher evaluation. Findings suggest several areas of concern, such as undesirable instructional changes and reduced teacher morale. Other potential problems include inconsistency of teacher value-added scores across different classroom contexts, the potential for misuse of new data systems, the complexity inherent in constructing measures of teaching “effectiveness,” and the role that school and district conditions play in influencing teachers’ responses to evaluation. We offer suggestions for improving the quality of teacher evaluation systems, including adopting multiple measures that incorporate information about practice, designing tests to make them more resistant to narrowing and score inflation, balancing group and individual accountability, and providing supports for instructional improvement.
Significance
Test-based teacher evaluation is widespread, and states and districts are working to develop evaluation systems that promote desired outcomes such as improved teacher retention and improved student learning. The findings and recommendations presented in this paper provide guidance for those who develop such systems and those who are affected by them.

Author

• Laura S. Hamilton, RAND Corporation