What’s in a Design? A Quasi-Experimental Evaluation of New Orleans High School Turnarounds

• In Event: Lifting All Boats With a Multisector Approach: Lessons From New Orleans

Abstract

As detailed in the previous abstract, the RSD pursued both district-led and charter-led turnarounds. The observational study described above found significant qualitative differences in the design and implementation of turnaround in these two sectors but can say little about the impact of these differences on student academic outcomes. In this presentation, we describe the differences in student academic outcomes seen in the charter- and district-led turnaround efforts. In this analysis, we compare the four redesigned high schools to RSD charter schools in 2007-8 and 2008-9 using a traditional value-added approach as well as a cross-sectional propensity score matching comparison.

The value added model we estimate is a simple linear model with school fixed effects. In this model we regress a student’s math or English score on the student’s previous year score in both subjects, student characteristics (e.g. gender, race, age, previous year participation in the free lunch program, dummies for gifted and special education students, previous year school attendance rate, test score school averages of the students previous year school), and a school fixed effect. We interpret this fixed effect as a measure of the educational value a school added in one year, while conditioning on student achievement and student and school demographics.

The value added analysis will not produce a causal estimate of the educational quality differences between the charter and district led turnarounds if the student body characteristics between the two types of schools are quite different. Because we found that the charter-led transformations displayed significantly different student body characteristics, we also explored a propensity score matching comparison, which ultimately provided a more balanced comparison between the charter and district-led schools.

The impetus of the turnaround movement in the RSD was to provide traditionally low-performing and under-served students a new and improved educational opportunity. We find that the charter-led turnarounds show significantly better results with the simple value-added analysis. However, these differences disappear in the propensity score matched sample suggesting that these differences may be an artifact of the student populations. While the above study finds significant qualitative differences between the charter-led and the district-led turnarounds, it may take more than a year or two before the charter-led turnarounds show significant improvement gains for similar students. This study is also limited to using students’ test scores to measure the differences in educational quality between the two types of schools. It may be that the improved school culture in the charter-led turnarounds created a safer and more comfortable learning environment for students, which would create significant, but unmeasured, differences in non-achievement outcomes (e.g., motivation, self-esteem, etc). It may also take time for the families of the lowest-performing students to make the transition from the traditional neighborhood schools to a charter school. A study is currently under way to evaluate the long-term benefits of charter-led versus district-led turnaround schools.

Authors

• Andrew McEachin, North Carolina State University

• Dominic J. Brewer, University of Southern California