Validating the Arizona Growth Model Using Educator Peer-Review Data: Does Value-Added Match Value-Observed?

• In Event: Value-Added in Arizona: Specifics, Statistics, and Impact

Abstract

Purpose

This study compared the results of Arizona’s value-added assessment and peer-review-based measures of effectiveness, providing a juxtaposition of value-added and peer-review methods of measuring educator impact on student achievement. It is the first in a series of explorations into the validity of the Arizona Growth Model as a valid value-added measure of educational effectiveness.

Perspectives

The use of “value-added” as a measure of teacher effectiveness and accountability is particularly attractive to many states and local education agencies (LEAs) because it is based on individual student growth, relatively easy to explain to parents and other community stakeholders, and cutting edge (i.e., “sexy”). For those states and LEAs implementing value-added analysis, however, choosing a single growth model to use as the foundation for value-added can be a challenging endeavor. Model characteristics are not always fully explained and users of the models (i.e., district or state personnel) may not be as thorough and diligent as desired when not only selecting but implementing their models at scale (Harris, 2011). Thus, there is a need within the research field to scrutinize these models more completely and assess their validity in particular.

Methods

Within Arizona, students are tested yearly from grades two through ten using two standardized student achievement tests, the Stanford Achievement Test, version 10 (SAT-10; Pearson, 2011) and the Arizona Instrument to Measure Standards (AIMS; Arizona Department of Education, 2010). AIMS is administered in grades three through eight and ten, and SAT-10 is administered in grades two and nine. Scale scores from these tests are analyzed with the student growth percentile package (Betebenner, 2011) to calculate yearly student achievement growth percentiles (SGPs). Principals are assessed using the Vanderbilt Assessment of Leadership in Education (VAL-ED) 360° assessment (Discovery Education, 2011) and teachers are observed using the TAP System for Student and Teacher Advancement rubric (National Institute for Excellence in Teaching, 2011).

Results

School median SGPs, aggregated using data from full-academic-year (FAY) students only, ranged from 20 to 77, with the majority between 40 and 60. Strong relationships (i.e., r’s > .5, all p’s < .05) were found when school median SGPs were correlated with teacher observation scores (see Table 1). Teachers were also grouped categorically according to their observation scores to explore whether thresholds within the observations scores contribute to the overall relationships between teacher observations and school median SGPs. In addition, similar analyses were conducted using teacher-level median SGPs and principal effectiveness observations, both contributing to overall findings regarding the relationships between value-added and observational evaluations of educator effectiveness.

Significance

The presence of strong positive relationships between observations of effectiveness and student achievement growth at the school level is encouraging when examining the validity of student growth models for value-added analysis. Results from this study should contribute to the greater dialogue on the design and implementation of education accountability systems, including the utility of student growth and value-added methodologies as part of this movement.

Authors

• Sarah A. Polasky, Arizona State University

• Wendy Miedel Barnard, Arizona State University