Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Committee or SIG
Browse By Session Type
Browse By Keywords
Browse By Geographic Descriptor
Partner Organizations
Search Tips
Personal Schedule
Sign In
“Teacher evaluation issues cannot be studied in isolation. Societal, school system, and school-level factors all influence the design of teacher evaluation policies…. Without an adequate understanding of the range of factors involved, and the ways they influence the impact of teacher evaluation policies, there is a risk of developing ineffective approaches to teacher evaluation” (pp. 3-4). – OECD (2009)
This study investigates the influential factors associated with the implementation of value-added modeling (VAM) as a method of teacher evaluation in the U.S. and other OECD countries. This research is relevant to comparative and international education in that it aims to identify and compare the key factors (demographic, economic, student achievement, graduation rate, dropout rate, teacher salary, labor union and teacher union membership rate, and political factors) that influence a state’s or a country’s decision to employ VAM as an assessment tool.
Value-added modeling (VAM) assesses a teacher’s effectiveness based on a comparison of current test scores of his or her students to the scores of the same students in previous years and to the scores of other students in the same grade. The aim of this comparison is to determine the value or contribution of the teacher in a given year, thereby providing a measure of teacher performance that can be compared with that of other teachers. The improved statistical methodology and availability of data led to the development of advanced value-added models (Raudenbush & Bryk, 1986), which in turn led to the use of such teacher assessment in some states in the U.S. (e.g., Tennessee, North Carolina, and Ohio) and some other OECD countries (e.g, United Kingdom and Australia) (Downes & Vindurampulle, 2007; Hibpshman, 2004).
The theoretical framework underlying this research is based on OECD’s (2009) framework for analyzing the factors that influence the design of teacher evaluation policies that contribute to the improvement of student outcomes through enhanced teaching performance and improved teaching practices. According to the OECD (2009), not understanding such factors and their influence on teacher evaluation policies can result in ineffective approaches to teacher evaluation.
There is considerable variation in the use of test scores and other components in the evaluation of teachers across countries, as well as within the United States (Close et al., 2018; Kraft & Gilmour, 2017; TALIS, 2009). The relative value placed on sometimes competing principles of Autonomy vs. Accountability, uniformity vs. creativity, objectivity vs. subjectivity, may have clear connections to the political, economic and demographic power structures within a nation or state (e.g., Brighthouse, Ladd, & Loeb, 2016). The salience of any of these particular values is likely a function of the characteristics of a community, such as trust and social cohesion.
We use OECD (2018) data on the components of teacher evaluation systems based on student test scores for the OECD countries that take the PISA exam, paired with data on economic, political, demographic, and educational performance to assess the degree to which each component is associated with measures of democratic practices, economic inequality, etc. Sources for the OECD data included the Indicators of Education Systems (INES) programme, the Teaching and Learning International Survey (TALIS), and the Programme for International Student Assessment (PISA). The primary analysis employs a multinomial logistic regression form to assess the relative associations between the various national characteristics and the likelihood of including select components of teacher evaluation systems (e.g., analysis of student test scores, repeated classroom observation, etc.). Using data provided by Close, Amrein-Beardsley, and Collins (2018), we then mirror the analysis of teacher evaluation systems based on student test scores (VAM) within the United States at the state level where comparable economic, political, demographic composition, and education performance might be associated with variation in the utilization of alternative means of teacher evaluation. The states vary in a number of aspects such as income level, rural/urban area, and race.
The research findings presented in this paper contribute to the literature on the design and implementation of teacher appraisal based on value-added modeling (VAM). Comparing states in the U.S. and other OECD countries with regard to the factors that influence their use of VAM may lead to a better understanding of the role of societal, school system, and school-level factors in implementing a teacher evaluation system based on student achievement as evidenced by student test scores.