The Impact of Achievement Measures on Estimates of Teacher Effects

• In Event: Excellence in Education Research: Early Career Scholars and Their Work

Abstract

Value-added modeling (VAM) has gained popularity among policymakers in recent years due to its alignment with the Federal initiatives to measure and evaluate school and teacher effectiveness based on their estimated contributions to student test scores. However, a growing number of studies found that the VAM scores tend to be sensitive to the statistical model (Rowan, Correnti, & Miller, 2002), the years of data (Koedel & Betts, 2011), and types of achievement measure or subscales of the same measure (Lockwood, McCaffrey, Hamilton, Stetcher, Le, & Martinez, 2007). Other studies found that teacher effects estimated from a sophisticated VAM model can be biased due to non-random sorting of students to teachers and classes (e.g., Rothstein, 2009; Newton, Darling-Hammond, Haertel & Ewart, 2010; Koedel & Betts, 2011). The systematic difference in student composition is a significant source of persistent differences in achievement gains that VAMs would wrongly attribute to teachers and schools, which is known as the student sorting bias (Rothstein, 2009). The one study that explored the sensitivity of VAM scores to achievement measures (Lockwood, McCaffrey, Hamilton, Stetcher, Le, & Martinez, 2007) was conducted based on the data of 70 mathematics teachers from one district. This study intends to address this problem by drawing on the Measures of Effective Teaching (MET) data of about 2,500 fourth- through ninth-grade mathematics and ELA teachers from six large school districts during 2009-2011. Specifically we will explore the extent to which teacher VAM scores are associated with student composition characteristics and prior ability, teacher classroom practice, and teacher working condition. We will examine if these correlations vary by the skills measured by achievement tests, the format, and other properties of the measures.

Author

• Ning Rui, Westat