Using Value-Added Models for Evaluating Teacher Preparation Program Effectiveness

• In Event: Value-Added Use: Applied Research to Inform Educational Practice and Policy

Abstract

Purpose

The purpose of this study was to examine the effectiveness of one of the U.S.’s largest Research I teacher preparation programs. Specifically, researchers conducted this study to investigate the impact of graduates from the large-scale teacher preparation program and graduates’ abilities to promote student learning and achievement in PreK-12 settings.

Perspectives

In response to the current policy climate emphasizing increased educational accountability in both PreK-12 and higher education, many teacher preparation programs have begun using value-added models to attempt to isolate the amount of student growth on standardized tests that might be attributable to their teacher graduates, and thus attributable to the quality of their preparation programs. The major assumption is that teacher-level value-added growth might serve as a reflection of the quality of training teacher graduates receive (Mellor, Lummus-Robinson, Brinson, & Dougherty, 2010).

Methods

Researchers compiled records for 187 teachers to investigate the relative growth of its graduates versus other teacher preparation program graduates. Given the state’s lack of a data system linking students to teachers, and teachers to their teacher preparation programs, collecting these data was in itself an issue. Researchers instead accessed the necessary data directly through the local schools districts with which they already had partnerships. For each analysis, location of training (i.e., the teacher preparation program of interest versus all other teacher preparation programs) served as the independent variable and teacher-level median student growth percentiles (MGPs) served as the dependent variables (Betebenner, 2009, 2011). Average MGPs were analyzed in addition to subject-specific MGPs (i.e., mathematics and reading). Correlation analyses were performed, as were analysis of variance procedures to evaluate mean student growth differences.

Results

Results evidenced very little relationship between teacher preparation program and student growth. For example, when teacher preparation program affiliation and teacher-level growth were correlated, the resulting coefficients were very small, ranging from r = 0.04 (program affiliation and average MGP) to r = 0.12 (program affiliation and mathematics MGP). Furthermore, all group comparisons lacked statistical significance and were associated with effect sizes and confidence intervals that suggested little to no practical or evaluative significance.
 
Significance

Teacher preparation programs are increasingly being held accountable for the student outcomes that can theoretically be attributed to their graduates (Author(s), in press; Coggshall, Bivona, & Reschley, 2012; Crowe, 2010). Although several large-scale projects are currently being implemented, little work has been done to help programs satisfactorily evaluate their effectiveness using value-added methodologies. This is largely due to major methodological issues still at play, as evidenced here and elsewhere (e.g., lack of randomization across programs, schools, and classrooms; Author(s), in press). These are also the major methodological issues that are often overlooked when conceptualizing such research, especially if and when for accountability purposes (Boyd et al., 2006; Darling-Hammond, 2006; Hamel & Merz, 2005; Harris & Sass, 2007; Russell & Wineburg, 2007). More research and dissemination is certainly needed to better inform the stakeholders calling for this research about the methodological issues, and in some cases impossibilities, associated with using value-added designs for such purposes.

Authors

• Kerry Chase Lawton, Arizona State University

• Audrey Amrein-Beardsley, Arizona State University

• Wendy Miedel Barnard, Arizona State University