The Impact of Student Growth Data on School District Choice: a Survey Experiment

• In Event: Populism, Privilege, and the Complex Purposes of Public Education

Abstract

We conduct an online survey experiment to assess whether the provision of different types of information about educational performance affects residential and school district preferences. Subjects engage in a simulation in which they are asked to imagine themselves as parents moving to a new city. Subjects then indicate their preferred choice between the five largest school districts in the metro areas of the five largest US cities. To guide these decisions, all subjects receive information on the demographic characteristics of the districts. In addition, some subjects are randomly assigned to receive some form of education performance data: either average student achievement, average student growth, both, or neither.

Research Questions
1. Compared to receiving no information about student performance, what are the effects of receiving average achievement data, average growth data, or a combination of both average achievement and average growth data on the racial and socio-economic compositions of subjects’ hypothetical school district choices?

2. Compared to receiving only average achievement data, what are the effects of receiving average growth data or a combination of both achievement and growth data on the racial and socio-economic compositions of subjects’ hypothetical school district choices?

3. Do these effects vary by subjects’ demographic characteristics?

Methods
For district-level educational and demographic data, we use the Stanford Education Data Archive 2v2.1. Achievement is measured such that a score of six represents a school district where the average student scores at about the same level as the average sixth grader in the national reference cohort. Growth is measured such that a score of 1.2 represents a school district in which the average student’s test scores improve about 1.2 grade level equivalents in one year.

Our experiment consists of a simulation embedded in an online survey. We used Amazon’s MTurk service to recruit 2,500 US adults. Subjects are asked to imagine that they are parents moving to a new city. They are instructed that, when deciding where to live, one of their top priorities is to choose a district for their elementary school-age child. The survey then provides basic demographic information for the five largest school districts in the metro area (median household income, the percentage of students eligible for free and reduced price lunch, and the racial composition of the study body). In addition to the demographic information, subjects are randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) a combination of both average achievement data and average growth data, or 4) neither. Based on these data, subjects choose their preferred school district. This process is repeated for the metro areas of New York, Los Angeles, Chicago, Houston, and Phoenix.

Results
Compared to subjects that receive no student performance data, subjects that receive average achievement data tend to choose even whiter and wealthier districts. This is consistent with the strong relationship between student demographic composition and average achievement.
The relationship between student demographic composition and average growth, on the other hand, is more diffuse. There are many high growth districts that serve a large proportion of less advantaged students. Subjects that receive only average growth data tend to choose notably less white and less wealthy districts.

Subjects that receive both average achievement data and average growth data tend to choose districts with similar demographic compositions as their peers in the control group. However, these subjects tend to choose districts that are less white and less wealthy than their peers that only receive average achievement data.

The largest effect sizes occur in metro areas where there is considerable demographic variation between districts and where there is a relatively high growth district that serves a relatively low-income and/or non-white student body.

We also find evidence of treatment effect heterogeneity by family income. The effects of providing growth data tend to be larger for subjects with a family income above $75,000. This pattern is encouraging for those that hope that the provision of education performance data that more accurately captures the effectiveness of schools (rather than the characteristics of the students themselves) can induce socially desirable choices.

Authors

• David Houston, Harvard University

• Jeffrey R. Henig, Columbia University