Value-Added "Bias" for Teachers of English Language Learners in Structured English-Immersion Classrooms

• In Event: 65.032 - Value-Added Models: Applied Research to Inform Educational Practice and Policy

Abstract

Purpose

Educational evaluation systems are increasingly aligning with Race to the Top (U.S. Department of Education [USDOE], 2013) and similar policies by incorporating value-added or student growth models (i.e., VAMs), in addition to traditional observation methods, for accountability purposes. The validity of the output derived via these evaluation systems, however, is questionable given the complexity of the calculations and the plethora of variables that may not be controlled for when calculating value-added (Amrein-Beardsley, 2014; Capitol Hill Briefing, 2011; Harris, 2011). Accordingly, this study accessed teacher evaluation scores to examine the extent to which teachers of English Language Learners (ELLs) in Arizona’s Structured English Immersion (SEI) classrooms demonstrated valid, or non-biased, value-added.

Perspectives

Current legislation in Arizona (ARS§15-03(A)(38); Arizona Department of Education [ADE], 2011) requires districts to use a “holistic” system of teacher evaluation (Darling-Hammond, 2013), including multiple teacher observations and student achievement data as key components. Concurrently, a separate state law (ARS§15.756.01; ADE, 2008) requires ELL students to receive four hours of Structured English Immersion (SEI) each day until proficiency is reached on the state language assessment. For teachers of SEI classrooms, however, the average student growth on the state mathematics and English/language arts (ELA) tests is used to calculate their value-added scores. Both mathematics and ELA are included in the evaluation because the nature of language learning is relevant to both subjects. Likewise, little empirical research has been conducted on the impact of using such value-added scores for teacher evaluation with high concentrations of ELL students within classrooms (Collins, 2012; Newton, Darling-Hammond, Haertel, & Thomas, 2010), in particular on the bias that is created in such classrooms. Examining this was of central interest in this study.

Methods

The data set included 1,845 elementary school teachers (52 schools, 9 districts) responsible for instruction within SEI and non-SEI classrooms. Data points included teacher observations and statewide achievement growth scores (average median Student Growth Percentiles [SGPs]), which were used to compare the performance of teachers responsible for SEI and non-SEI classroom instruction accordingly.

Results

Teachers of SEI (n = 108) and non-SEI (n = 1,737) classes did not differ in their observation scores; however, an independent samples t-test revealed differences in the mean value-added scores of teachers (t(726) = 3.42, p < .01; see Table 1 - removed as submission system incompatible).

Significance

Recent research has found the SEI model to be an impediment to students’ access to equitable learning opportunities in comparison to their non-ELL peers (Lillie, Markos, Arias, Wiley, 2012; Lillie et al, 2010). This study also demonstrates that teachers of SEI classrooms have less opportunity to demonstrate value-added, implying issues with bias (see also Newton et al., 2010). Use of the SEI model to support student growth may need to be re-examined based upon the results of the current study, particularly given results indicating unfairness for both students and teachers. Alternately, schools might consider using data from the state language assessment (i.e., the Arizona English Language Learner Assessment [AZELLA]), as this might better serve as the measure of teacher effectiveness in developing students’ language proficiency.

Authors

• Sarah A. Polasky, Arizona State University

• Ann Walker Nielsen, Arizona State University