Neighborhoods as a Level: Decomposing Variance in Urban Student Outcomes in Philadelphia and New York City

• In Event: Housing, Neighborhoods, and the Schools Children Attend: Policy Evidence

Abstract

How much of what looks like a school effect is really a neighborhood effect? In most research on school performance, neighborhood enters the model, if at all, as an observed covariate added to a two-level specification. Bryk and Raudenbush (1988) argued that neighborhoods should instead be modeled as a formal clustering level, but the broader literature has not followed. Nieuwenhuis and Hooimeijer (2016) find that across 88 studies, neighborhood effects are modeled almost exclusively as predictors of individual outcomes, not as a structural level that partitions variance. Moerbeek (2004) shows analytically that omitting a clustering level redistributes its variance to the remaining levels, inflating school-level estimates. However, this has not been demonstrated in a specific urban district, nor has anyone tested whether census covariates suffice to absorb neighborhood structure.

Consider Upper Kensington and Graduate Hospital, two Philadelphia neighborhoods. Both contain schools serving predominantly Black and Hispanic students, yet poverty rates differ dramatically (51.5% vs. 10.6%), as does average ELA proficiency (17% vs. 40%). A model that ignores neighborhood context attributes those differences entirely to schools. Yet neighborhoods are rarely available in administrative datasets. The administrative boundaries that are readily available (such as districts, catchment zones) are governance units, not community units. The distinction matters because although census covariates can capture mean differences across places, much of what makes a neighborhood a neighborhood are unobserved features like social capital and the accumulated institutional history of a place (Sampson, Morenoff, & Gannon-Rowley, 2002).

Using public data from the School District of Philadelphia and the American Community Survey, we model school-year observations as nested within schools within neighborhoods, examining ELA proficiency, math proficiency, attendance, and chronic absenteeism. We compare conventional two-level models with three-level models that partition variance across time, schools, and neighborhoods, then add school composition, institutional characteristics, and ACS measures to test how much neighborhood structure is absorbed by observed covariates. We verify the pattern against Moerbeek (2004) and use a simulation calibrated to the Philadelphia design to examine how school misclassification varies with covariates. In unconditional models, neighborhoods account for 36 to 47 percent of achievement variance in Philadelphia but only 10 to 14 percent of attendance variance. When neighborhoods are omitted, the school-level ICC is inflated by the amount Moerbeek (2004) predicts, within one percentage point. Even after full covariate adjustment, residual neighborhood structure is large enough to change the relative standing of 7 to 19 percent of schools in our sample.

The pattern replicates in New York City, where neighborhoods explain 41 to 44 percent of achievement variance. We focus on two implications. First, neighborhoods matter as a level of educational organization that shapes how schools perform and how they are compared. Second, modeling neighborhoods in urban school research is feasible with public data alone, and we show how. We extend the framework to New York City using NYC open data and Neighborhood Tabulation Areas created by the Department of City Planning, and show how the approach generalizes to any urban district with publicly available neighborhood boundaries.

Author

• Katherine Jane Strickland, University of Florida
  Presenting Author