Methodological Concerns and Challenges With Evaluating Teacher Education Programs

• In Event: 52.053 - Learning From the Federal Market-Based Reforms: Lessons for the Every Student Succeeds Act

Abstract

Considering the Higher Education Act (HEA, 2008), and the Every Student Succeeds Act (ESSA, 2016), stronger accountability proponents have been requiring teacher educators to evaluate their programs and/or be evaluated by others (e.g., state departments of education). The goal is to assess to what extent programs are preparing measurably effective teachers, with measurable effects often being defined as teachers’ purportedly causal impacts on student achievement over time (see also Cochran-Smith, 2009).

The theory is that by holding teacher educators accountable, teacher quality will increase, and student achievement will improve. Not surprisingly, however, how such evaluations might be conducted is not nearly as straightforward as some (e.g., said policymakers) might assume (see also Boyd, Grossman, Lankford, Loeb, & Wyckoff, 2009). Hence, the first purpose here is to organize this work via a conceptual framework developed in partnership with stakeholders. The second purpose is to measure the value-added of one of the largest teacher education programs in the U.S. all the while making transparent the major (un)anticipated issues addressed along the way.

Approximately 100 peer- and editorially-reviewed articles were studied to collaboratively develop a conceptual framework. Second, teacher-level value-added data was collected from a convenience sample of graduates/non-program graduates (dichotomously coded), all of whom were teaching in the seven large school districts with which the teacher education program had ongoing partnerships. The group arrived at two sets of principles.

First, teacher educators should: (1) conceptualize the purposes for evaluations; (2) define effective teacher education programs and teacher graduates; (3) build valid evaluation models; (4) resolve whether and which standards might be used to structure these models; (5) choose appropriate evaluation methods; (6) decide who should be involved in decision-making processes; and (7) determine how these program evaluations might be financed and supported (see also Author(s), 2014).

Second, the major issues they still encountered had to be made transparent. These issues included, for example, data inadequacies given the state had not (and still has not) connected teachers with their graduating institutions, related sample size issues, the non-random assignment of students to teachers’ classrooms and teachers to teacher education programs, interfering variables that also biased the causal estimates to be made, and the like (see also Author(s), under review).

This research provides a framework to inform others regarding how they might begin a similar process, using the aforementioned seven key imperatives, major issues notwithstanding. Accordingly, this exploration also clearly illustrates how very difficult it is to conduct this research, regardless of the preparatory work. While results of this study did not provide quantitative evidence that graduating from the program of interest was related to student growth greater than graduating from other teacher education programs throughout the state, results did yield and make more transparent many of the major issues to be faced (that others will also likely face).

Authors

• Audrey Amrein-Beardsley, Arizona State University

• Tirupalavanam G. Ganesh, Arizona State University

• Kerry Chase Lawton, Arizona State University