The Impact of Intensive Partnerships for Effective Teaching on Student Outcomes and Teacher Allocation

• In Event: 46.048 - Using Measures of Teaching Effectiveness to Improve Classroom Teaching and Student Outcomes: The Intensive Partnerships for Effective Teaching After Three Years

Abstract

This paper summarizes two lines of research on the impact of the IP reforms. First, we report the overall impact of the reforms on student outcomes, using a quasi-experimental method. We also present trends in the allocation of effective teaching to low income and minority students (LIM) within the sites.

In both lines of research, we seek to describe changes in outcomes that are associated with the implementation of the IP reforms at the sites. The overall impact estimates use a difference-in-difference method based on publicly available state-wide school-level data, which provide causal estimates under plausible assumptions. The pre-post differences in outcomes for IP schools are compared to those for a group of schools selected using propensity score estimation to restrict the samples to a common support and analyzed using multivariate models to control for covariates. We examine reform impact on state assessment test scores as well as on a variety of other outcomes, such as graduation and attendance rates, when they are available from state sources. We calculate significance in two different ways. The more conservative method uses a permutation estimator, resampling the data to get an estimate of confidence intervals that does not rely on distributional assumptions. We also provide confidence intervals using a standard variance components estimator.

For two of the sites, the post-reform outcomes are in general higher than those of their comparison groups of schools. These differences are mostly significant, using the standard variance components estimator. Of note, in several cases, the better outcomes in the treatment sites started a year or two prior to the official beginning of the reforms, possibly indicating an anticipatory effect or an inadequate control for confounders. In the other sites, we do not find any significant positive impact.

In the other line of research, we use site-provided data to estimate value-added for 4th-8th grade math and reading teachers for two years prior to the reforms and four years following. We use this value-added estimate as a common measure of teacher effectiveness for all sites and years. Our pre/post analysis examines the correlation between teacher value and LIM status, overall as well as between and within schools, in each site in each year.

We find that LIM students on average have teachers who are at least as effective as other students. This was the case prior to the reforms and has remained so. The overall differences in average effectiveness between LIM students and others are not big and we find that in some cases the greater allocation of effective teachers to high LIM schools is offset by within-school sorting that, on average, allocates less effective teachers to LIM students.

These two lines of research, when combined, suggest that the IP reforms to measure teacher effectiveness and use those measures for personnel decisions have the potential to improve student outcomes. However, it is unlikely that reallocating teachers based on value added to high LIM schools will have a substantial impact on achievement gaps based on race or income.

Authors

• John Engberg, RAND Corporation

• Paco Martorell, RAND Corporation

• Jennifer L. Steele, American University

• Matthew Baird, RAND Corporation

• Ethan Scherer

• Italo Gutierrez, RAND Corporation