Site Selection in District Research: Comparing Measures of Effectiveness Using Hierarchical Longitudinal Growth Models of Performance

• In Event: Studying School District Central Offices: Examining Issues of Methods, Research, and Context

Abstract

Purpose:
The purpose of this study is to present and discuss issues of site selection for district research from all possible districts in a state focusing on how best to identify where in the distribution of “effectiveness” any one district may be in relation to all other districts in the population as a means to provide contrasts and generalizable findings across districts that may be outperforming, underperforming or at the norm. To date, much of the literature on the effectiveness of school district central offices has focused on in-depth qualitative studies of the actions of central office personnel and constituent school leaders (Honig, 2009; Rorrer, et al., 2008). However, few of these studies link findings to measures of district effects on student achievement. One reason is the issue of site selection and the justification of labeling a district as “effective” (Bowers, 2010). Recently, Bowers (2010) proposed and tested a new method for site selection, examining all districts in Ohio over seven years using hierarchical growth modeling of test scores across subjects and grade levels, identifying districts that significantly outperformed districts at the norm while controlling for background characteristics. Following the recommendations from the literature on school effects (Cuban, 1984; Heck & Moriyama, 2010) this work provided a means to identify significantly unusual districts from comprehensive longitudinal datasets. The purpose of the present study is to extend this model to take into account additional factors including the amount of variance between schools within a district.

Framework:
Past district site selection models have failed to consider the variance between schools within districts. This is an important consideration given that district effects can be heavily influenced by individually high performing schools within a district. The aim of the present study is to test an extended longitudinal growth model of student achievement controlling for background variables and between-school variance within districts. Outperforming districts would thus have significantly higher growth in scores than comparison districts with similar demographics, and would do so more evenly across their schools.

Methods and Data Sources:
The model will be demonstrated using the Ohio Department of Education school performance database, analyzing all districts in the state over ten years, 2001-2002 through 2011-2012, n=613. The dependent variable is the school Ohio Performance Index Score with a three level hierarchical linear growth model nesting time points per school within schools within districts controlling for student demographics.

Results and Significance:
As will be fully detailed in the final paper, districts that significantly outperformed predicted gains in achievement, while controlling for background variables and between-school variance, are identified as potential sites for further study in comparison to districts at the norm and districts that underperformed. This study significantly adds to the research on districts through providing a means to more accurately identify and compare all districts in a state through providing the ability to contrast findings from qualitative studies between districts knowing where each district sits in the distribution of effectiveness for an entire state.

Author

• Alex J. Bowers, Teachers College, Columbia University