Just Keep Swimming: Implementation of the Intensive Partnership Reforms

• In Event: 46.048 - Using Measures of Teaching Effectiveness to Improve Classroom Teaching and Student Outcomes: The Intensive Partnerships for Effective Teaching After Three Years

Abstract

The IP reforms have emphasized five key levers, or aspects of the IP theory of action – teacher workforce conditions, teacher evaluation policies, professional development (PD), strategic compensation, and career ladders. In this paper we examine the status of implementation of the five levers across the seven IP sites, describe teachers’ and school leaders’ responses to these reforms, and identify factors that appear to hinder or facilitate effective implementation.
The paper draws on several data sources, with each activity carried out annually: central office staff interviews, document reviews, teacher surveys (N=4,697; response rate = 77% in 2013), school leader surveys (N=1,172; response rate = 72% in 2013), and school visits (3-4 per site each year) that included teacher and school leader interviews as well as teacher focus groups. For qualitative data, we developed a thematic coding scheme and analyzed the data according to role group (teacher, school leader, central office staff) and key theme (e.g., opinions about evaluation measures). Survey data were weighted for sampling and nonresponse. Analyses for this paper consist primarily of frequency distributions and crosstabs that allow us to compare responses by teacher and school characteristics.
The findings indicate that four years into the initiative, all sites have implemented a combined effectiveness measure as well as reforms to improve teacher recruitment, hiring, and placement. Sites’ combined measures include classroom observations, student outcomes, and other measures such as student perception surveys, though six sites are struggling with how best to measure student growth for teachers in grades and subjects that do not currently have state tests. Most sites have only begun to implement other IP reforms—e.g., compensating teachers based on effectiveness and creating career ladders. Although leaders in all sites indicated that customizing PD based on evaluation results was a priority, sites struggled to determine what type of PD would meet teachers’ needs, and to track PD participation in a way that would allow them to gauge its effectiveness. Addressing teachers’ PD needs is crucial to achieving the IP goal of ensuring effective teachers in every classroom.
Teachers and school leaders expressed mixed opinions about the validity of the evaluation measures. Majorities of both groups reported that the classroom observations provided valid information about teaching, whereas opinions about validity were much less positive for the measures based on student achievement and student perceptions. In addition, most teachers reported having access to PD opportunities that help them improve their teaching, but some reported constraints on participation stemming from funding or scheduling.
Pre-existing practices, policies, and organizational cultures were associated with implementation and educators’ responses. For example, in one site that had initially emphasized the use of observations to support professional learning rather than as an element in an evaluation system, teachers were more supportive of the observations than teachers in other sites. One key challenge sites faced is reductions in general operating funds, which have led most sites to alter their implementation plans, extend implementation timelines, and seek additional funding for the reforms.

Authors

• Laura S. Hamilton, RAND Corporation

• Deborah J. Holtzman, American Institutes for Research

• Elizabeth Steiner, RAND Corporation

• Abby Robyn, RAND Corporation

• Jeffrey M. Poirier, American Institutes for Research