Setting Targets for School Improvement on Social-Emotional Learning and Other Nonacademic Indicators

• In Event: 47.090 - Social-Emotional Learning Measurement and Continuous Improvement: Lessons From the CORE Districts

Abstract

Examining the statistical properties of measures of students’ SEL is a foundational step towards designing robust systems for assessing school quality that incorporate a range of measures. Another central question is how those measures are used in school quality systems—in particular, how setting targets around the included measures affects how schools perform on those measures. Although there is a robust body of literature studying targets for academic indicators within school quality systems (Ladd & Walsh, 2003; Richards & Sheu, 1992), few studies explore target setting for non-academic indicators. Focusing on schools within the CORE districts, we investigate how moving performance targets for non-academic indicators affects school quality ratings. We ask: (1) How does school performance on CORE’s School Quality Improvement measures vary across schools and over time?; and (2) How does the setting of targets on CORE’s non-academic indicators at various levels impact the number and types of schools that make progress toward or reach the target?

Using data from more than 800 elementary schools within the eight CORE districts, we focus on the index levels and California Accountability Dashboard Level (CADL) categories for 14 measures within the two domains of the CORE School Quality Improvement Index (SQII): four academic domain measures (i.e., academic achievement and growth in math and ELA); and 10 social-emotional learning (SEL) and school culture-climate (CC) domain measures, including suspension/expulsion rate, chronic absenteeism, English learner redesignation rate, four student SEL measures, and three school CC measures. To explore our first research question, we present descriptive statistics, correlation analyses, and transition tables. For example, we find that, as compared to the achievement measures, there is substantial year-to-year variation and more even distribution among the non-academic SQII measures.

To examine our second research question, we shift the thresholds for the CADL categories one index level unit to the left (i.e., narrowing the lowest-performing category (red)) and one index level unit to the right (i.e., widening the lowest-performing category; note that due to the negative skew in the suspension rate measure, we impose a 0.25 index level unit shift in lieu of a 1 index level unit shift.) Using these data, alongside school-level student demographic data, we examine how a one-unit shift impacts the number and types of schools that improve or worsen their CADL category. Preliminary results reveal that all measures are sensitive to even these small shifts in thresholds. Analyses of the student demographics in the schools that are most sensitive to these relatively minute changes are underway, but the preliminary results indicate that small schools are especially vulnerable.

Given evidence on the importance of non-academic outcomes and changes to education policy under the Every Student Succeeds Act, states have begun to include non-academic measures in their school accountability systems. Little is known, however, about how these indicators vary between schools and over time, or about what calculation models to use and where to set targets. Because these indicators are used to determine school status, setting purposeful and evidence-based policy is critical. The proposed study will provide initial evidence to this effect.

Authors

• Rachel Sue White, Old Dominion University

• Morgan S. Polikoff, University of Southern California

• Shira Korn, University of Southern California