Unpacking the Evidence: A Review and Reflection of Professional Preparation Evaluation Measures

• In Event: 21.035 - What Is the Impact of Teacher Education? Research on the Effectiveness of Completers During Induction

Abstract

Purpose
This study explores the relationships among measures of candidate readiness, candidate performance, candidate and employer satisfaction, and overall program effectiveness, and considers the impact of educator preparation evaluation measures among stakeholders. Measures are used to ‘rate’ and ‘rank’ educator preparation programs and can have an impact on recruitment, retention, moral, and achievement of the programs, faculty, students and stakeholders.

Perspective
In 2012 a Georgia task force convened to recommend a system of educator preparation evaluation measure. The Preparation Program Effectiveness Measures (PPEM) were developed as accountability measures to ensure EPP’s would meet program standards, maintain consistency, communicate results, and improve Georgia’s students’ learning. Performance measures include: edTPA scores, state content assessments, teacher evaluations, and employer and novice teacher surveys. An algorithm weights the measures to generate score and rating ranging 4-Exemplary to 1-Low Performing.

Consistent with a national movement pushing states to publish EPP “report cards” (U.S. Department of Education, 2016), the espoused purpose of the movement is to improve EPP’s and inform stakeholders (von Hippel & Bellows, 2018). However, research regarding the ability to discriminate between the impacts of teachers from EPP’s have been negligible (von Hippel & Bellows, 2018).

Considering the components of PPEM in Georgia, EPP’s need to review the validity and reliability of the individual measures and the impact of the dashboard as they negotiate internal conversations of continuous improvement and external conversations with stakeholders, leading to these questions:
1. How can we use PPEM to improve programs and school partnerships?
2. What changes might render the PPEM system more helpful?

Methodology
The study uses a mixed method design, drawing on both qualitative and quantitative data. Qualitative evidence was gathered from interviews and open-ended question surveys from EPP completers and employing principals. Quantitative evidence included the analysis of performance measures (edTPA and Content Assessments) and outcome measures (Intern Keys and Teacher Keys) were correlated from the completion and induction year. Data are currently being analyzed.

Results
One purpose of Georgia’s PPEM is to improve partnerships between preparation programs and schools districts. However, there has been no guidance on that nor any indication that school partners are even aware of the PPEM dashboard. Quantitative data and additional qualitative results are forthcoming.

Significance
The impact of the movement to rate and rank EPP’s is unknown. The PPEM dashboard provides opportunities to unpack components and reflect on their actionability—how can they contribute to an EPP’s continuous improvement? PPEM opens a door for collaboration with stakeholders to delve into indicators of concern/success. Should these reflections be fruitful, the efforts to build and maintain PPEM are warranted. If merely a branding exercise, given teacher shortages, ratings may exacerbate problems, especially in light of negligible discrepancies in previous studies (von Hippel & Bellows, 2018). In our quest for accountability are we doing more harm than good in recruiting and retaining impactful teachers in our most needed schools with the calculation and publication of PPEM report cards.
 

Authors

• Cynthia Bolton, Georgia Southern University

• Amee Adkins, Illinois State University