The Influence of Design Decisions on Identification of Comprehensive Support and Improvement Schools

• In Event: Sheraton Roundtable Session Three
  In Roundtable Session: Improving Low-Performing Schools Through Accountability Policy Under the Every Student Succeeds Act (Table 2)

Abstract

As a result of flexibility afforded to states under ESSA, states had to make many important decisions regarding how to identify CSI schools; and each decision potentially had consequences for which schools were identified as CSI. In this paper, we examine processes for CSI identification in three states with different approaches to implementing school accountability requirements under ESSA (California, Florida, and Ohio), in order to better understand the motivations behind different design choices as well as the influence of those choices on determining which schools are designated as CSI.

Methods – For each state, we interviewed state level education officials overseeing the state accountability systems to understand the process for of designing the accountability system, as well as the motivation and intention behind design decisions – in particular, the introduction of new or unique measures.

Following these interviews, we examined the number and characteristics of CSI schools in these three states to better understand which schools are identified by the states’ current policies. We also conducted simulations to understand the extent to which specific policy decisions regarding the identification of low-performing schools affect the set of schools identified as low-performing. We also used correlation and regression analyses to determine how distinctive each measure is from other measures in each state’s accountability system. Additionally, we calculated the predictive power of each accountability measure on CSI identification using logistic regression to determine which accountability measures are the strongest predictors of CSI identification.

Preliminary Results – Preliminary results indicate that state officials were guided by several motivations when designing their accountability systems, including: (a) a desire to move away from NCLB’s strong emphasis of student proficiency in ELA and Math by incorporating a more holistic set of measures, and (b) achieving balance in the kinds of schools identified as CSI.

We found that states’ efforts to reduce the emphasis on ELA and math achievement has influenced which schools were identified as CSI. In Ohio and California, measures of growth and year-to-year change were relatively distinct from measures of academic achievement in ELA and math, suggesting that the inclusion of these measures did influence the set of schools identified. Several non-academic measures in California, suspension rates in particular, were also quite distinct from other measures (Exhibit 1). Using simulations, we found that the inclusion of these measures did indeed have a sizable influence in determining which schools were identified as low performing (Exhibit 2). In Florida, however, we found strong correlation between ELA/Math achievement and other measures included in the state’s accountability system. We also found that the inclusion of measures less correlated with ELA/Math in Florida’s accountability system, (e.g. EL progress or College Career Acceleration), had minimal effect on how schools performed according to the state’s composite index and ultimately whether they were identified as CSI.

Significance – Given flexibility, states have approached identification of CSI schools in different ways making different design choices. States, however, have little guidance on how different design decisions might influence which schools are identified. This study attempts to fill that gap.

Authors

• Katie Hyland
  Presenting Author

• Drew Atchison, American Institutes for Research
  Presenting Author

• Umut Ozek, RAND Corporation
  Non-Presenting Author