Multimetric teacher accountability? Not when student test scores are included

• In Event: Globalizing Test-Based Accountability in Education (ii): Local Imaginaries, Recontextualisations, and Enactment Dynamics

Proposal

Testing is a core practice in education and central in the expanding calls for accountability worldwide. Trumpeted as an objective indicator of quality, student test scores are increasingly linked with high stakes outcomes for teachers and schools (Smith, 2014). The importance of test scores as a taken for granted notion of quality testing permeates all aspects of education, shaping the experiences of the actors involved. Teachers, as the front line providers of education, are positioned to feel the brunt of the pressure when student test scores are the valued outcome. The role of testing in teachers lives is part of the larger Global Testing Culture (Smith, 2016) which is fueled, in part, by powerful international organizations who promote the increased use of testing in multiple forms and at multiple levels (Addey & Sellar, 2018; World Bank, 2017).
Teachers are commonly regarded as the main actors within schools, contributing to and shaping student development and learning (Jimerson & Haddock, 2015). Over the past 30 years teachers have been held increasingly accountable for the quality of education in their classroom. During this transition, the line between teacher appraisals, traditionally an instrument for continuous formative teacher feedback, and summative teacher evaluations has blurred. Student test scores, as an ‘objective’ measure, are increasingly used in teacher appraisals in response to historic questions that evaluations are based on ‘subjective’ components. This has resulted in a trend toward performance-based pay, as seen in countries as diverse as Kenya, Portugal, and Israel (Barnes et al. 2016). In a recent review of World Bank knowledge products, performance-based pay was the most common teacher policy recommendation (Fontdevila & Verger, 2015).
Although there is an emphasis on including multiple measures in teacher evaluations, especially when high stakes are attached (Mellor & Griffith, 2015), the prominence of testing as the measure of quality suggests that not all components are given equal weight or seen as equally important in practice. This article explores the role of testing in high stakes teacher appraisal systems across 33 countries using data from the 2013 TALIS; addressing both the prominence of student test scores and their relative importance in teacher’s perceived feedback utility. Specific questions addressed in this study include:

1. How common are the use of student test scores and high stakes in teacher appraisals?
2. How much importance is placed on different components when teachers receive feedback from their appraisal?
3. How does teacher’s perception of feedback utility differ by the degree test scores are emphasized in their feedback?

Larsen’s (2005) definition of high stakes as present if results are “tied to increases in salary, promotion and maintenance of employment” (p. 296) is used to identify test-based high stakes teacher appraisal systems and a three level HGLM is used to examine the impact of test score emphasis on teacher’s perceived utility of feedback. Results indicate the test scores are the most common component used in teacher appraisals, regardless of the stakes, ranking just above teacher observations. Furthermore, student achievement – often expressed in test scores – is the most emphasized piece of feedback teachers receive from administrators. When emphasized, teachers are more likely to feel their appraisal had limited impact on their instruction and was completed solely as an administrative exercise.
The ubiquitous application of student test scores as the most common component in teacher appraisals is another example of the importance placed on these seemingly objective measures of education quality and part of the larger Global Testing Culture. The results on perceived feedback point to a troublesome pattern that mirror some of the undesirable, indirect effects found in other studies on test-based accountability (UNESCO, 2017). Additionally, as Delvaux et al. (2013) made clear, individuals that do not see value in feedback are less motivated and less likely to take action. This suggests that overemphasizing test scores in appraisal feedback can ultimately affect teacher’s self-esteem and student learning. Finally, this study speaks to some of the current debates around teacher accountability, including how states in the U.S. should design accountability under the recently passed Every Student Succeeds Act. Although the reliance on student test scores may have been reduced from the NCLB era (Darling-Hammond et al., 2016), our results suggest that the dominance of testing in the education quality dialogue means that, although conceptually student test scores may not dominate the accountability design, in practice they are often prioritized.

References

Addey, C. & Sellar, S. (2018, Forthcoming). Why do countries participate in PISA? Understanding the role of international large-scale assessments in global education policy. In Verger, A., Novelli, M. & Altinyelken, H. K. (Eds.), Global Education Policy and International Development: New Agendas, Issues and Policies. New York: Bloomsbury Academic.

Barnes, S.-A., Lyonette, C., Atfield, G,. & Owne, D. (2016). Teacher's Pay and Equality: A Literature Reivew. Longitudinal Research into the Impact of Changes to Teachers' Pay on Equality in Schools in England. Warwickshire, UK: Warwick Institute for Employment Research.

Darling-Hammond, L., Bae, S., Cook-Harvey, C. M., Lam, L., Mercer, C., Podolsky, A., Stosich, E. L. (2016). Pathways to new accountability through the Every Student Suceeds Act. Stanford, CA: Learning Policy Institute & Stanford Center for Opportunity Policy in Education.

Delvaux, E., Vanhoof, J. Tuytens, M., Vekeman, E., Devos, G., & Van Petegem, P. (2013). How may teacher evaluation have an impact on professional development? A multilevel analysis. Teaching and Teacher Education, 36, 1-11.

Fontdevila, C. and Verger, A. 2015. The World Bank's Doublespeak on Teachers: An Analysis of Ten Years of Lending and Advice. Brussels, Belgium, Education International.

Jimerson, S. R., & Haddock, A.D. (2015). Understanding the importance of teachers in facilitating student success: Contemporary science, practice, and policy. School Pscyhology Quarterly, 30(4), 488-493.

Larsen, M. A. (2005). A critical analysis of teacher evaluation policy trends. Australian Journal of Education, 49(3), 292-305.

Mellor, M. & Griffith, D. (2015). Multimetric Accountability Systems: A Next-generation Vision of Student Success. Alexandria, VA: ASCD.

Smith, W. C. (2016). The Global Testing Culture: Shaping Education Policy, Perceptions, and Practice. Oxford: Symposium Books.

Smith, W. C. (2014). The global transformation toward testing for accountability. Education Policy Analysis Archives, 22(116).

UNESCO (2017). Global Education Monitoring Report 2017/8: Accountability in Education – Meeting our Commitments. Paris, France: UNESCO.

World Bank (2017). World Development Report 2018: Learning to Realize Education’s Promise. Washington, DC: World Bank.

Author

• William Smith, UNESCO Global Education Monitoring Report