Measuring Stability in School Effects on Social-Emotional Learning (SEL): Evidence From Three Years of SEL Data

• In Event: 47.090 - Social-Emotional Learning Measurement and Continuous Improvement: Lessons From the CORE Districts

Abstract

Do schools influence students’ SEL development? If so, can we measure this influence in a reliable way? A critical step towards answering these questions is examining whether schools have a stable effect on students’ SEL over time. This paper investigates the stability of school effects on students’ self-reported SEL in three waves of the first large-scale panel survey of students’ SEL. Understanding the stability of estimates of school effects will provide fundamental insights into whether and how SEL measures might be used to assess school quality.

The study builds on previous work (Authors, date) using two waves of data that found substantive differences across schools in SEL growth, of magnitudes similar to those for academic achievement, suggesting that schools might contribute to students’ SEL. However, additional work is needed to warrant interpreting these differences as causal effects of schools on students’ social-emotional learning. This study takes a further step in this direction by investigating the stability of the estimated school effects over multiple years in order to help assess whether the estimated impact of schools are reliable. We will further estimate school effects for different subgroups of students, such as girls compared to boys, socioeconomically disadvantaged students compared to their more advantaged counterparts, and students of different racial and ethnic backgrounds; this paper will then assess stability of these subgroup estimates over time. Under the assumption that the factors that determine a school’s capacity to shape social-emotional skills are changing only slowly, stability of school effects across years would provide further evidence that schools contribute to students’ social-emotional development. If, however, school effects vary substantially across years, then external factors or measurement error are more likely to explain the school effects. Although stability is not a sufficient criterion to interpret school effects as causal, it would provide necessary evidence that schools can measurably contribute to students’ SEL development.

To answer these questions, we examine a unique panel dataset of nearly 500,000 student responses each year across three years within six large districts in California on a set of self-reported survey questions related to four social-emotional constructs: growth mindset, self-efficacy, self-management, and social awareness. We use methodology similar to that of conventional value-added models on standardized assessments to assess variation across schools for each construct. We do this separately for two years (2016-17 and 2017-18), always controlling for one year of prior SEL responses (i.e., 2015-16 and 2016-17, respectively) in order to examine students’ growth on a given SEL construct. We then compare the results of these two models across years. We compare the outcomes of this analysis to growth results on standardized assessments in math and English language arts for the same students, to provide a sense of the relative magnitude of schools’ effects on students’ SEL. Overall, this paper aims to build upon a growing body of evidence that schools can influence students’ social and emotional development and will provide critical evidence as to whether this influence is a true and stable effect over time.

Authors

• Hans Fricke, Stanford University

• Susanna Loeb, Brown University

• Robert H. Meyer, Education Analytics

• Andrew B Rice, Education Analytics Inc.

• Michael Christian, Education Analytics

• Libby Pier, Education Analytics

• Heather J. Hough, Stanford University