Measuring the Growth of Social-Emotional Competence in Elementary School Years using Behavioral Rating Scales

• In Event: Poster Session 03
  In Poster Session: PS 03 Topic - Prevention and Interventions

Abstract

Behavior rating scales are widely used in schools to assess student status over time. Yet, they are rarely interrogated for longitudinal measurement invariance, a requisite feature for their use to assess behavior change. This is particularly true in the measurement of social-emotional competence (SEC), the targeted outcome for social and emotional learning (SEL) interventions. In order to accurately measure student SEC growth over time, it is necessary to ensure that the same construct is consistently measured across occasions.
This study examines (1) the longitudinal factorial invariance of a widely used teacher-completed behavioral rating scale—the DESSA-Mini (Naglieri et al., 2014), and (2) the average SEC growth trajectory over two years, using data collected in the context of a district-wide implementation of Promoting Alternative THinking Strategies (Kusche & Greenberg, 1994). The analysis sample consists of 1,151 randomly-selected students from 15 schools (grades K-2 in Year 1; 48% female; 63% Hispanic/Latinx, 16% non-Hispanic Black, 14% non-Hispanic White; 88% low-income), assessed on the full 72-item DESSA in Fall (October) and Spring (June) for two consecutive years (2011-2012 to 2012-2013). This study focuses on 24 items that comprise three DESSA-Mini Forms (8 items each), which are intended to be used interchangeably as repeated measures across time. Within a year students were mostly rated by the same teachers, whereas across years they were mostly rated by different teachers. We first tested four levels of factorial invariance (i.e., configural, weak, strong, and strict; Widaman et al., 2010) of each DESSA-Mini Form, and then estimated the average SEC growth trajectory using second-order latent growth modeling, freely estimating Year 2 time coefficients (i.e., not assuming a linear growth across years). Full Information Maximum Likelihood estimator with robust standard errors (MLR in Mplus) was used for missingness and non-normality.
A series of model fit comparisons with increasing invariance suggest that at least strong factorial invariance (i.e., factor loading and intercept equivalence over time) can be assumed. We found inappreciable changes in practical fit indices (despite the significant chi-square differences that are sensitive to large sample sizes). The strong factorial invariance model showed an acceptable fit with each DESSA-Mini Form (see Table 1). With strong factorial invariance assumed, the latent means in Year 1 Spring and Year 2 were all significantly higher than the baseline. The average SEC growth trajectory showed non-linear growth throughout two years: significant growth in Year 1, a slight decrease or maintenance over the summer, and a significant but lower rate of growth in Year 2 compared to Year 1 (see Table 2).
Analysis with the DESSA-Mini raises the possibility that behavioral ratings scales can be used to assess growth of SEC over time, with evidence supporting that the same SEC construct is measured at each occasion. Under a district-wide SEL implementation condition, students showed growth in SEC, on average, within and across academic years despite a modest “summer slide.” Further studies with diverse samples, especially under experimental conditions, will advance our knowledge on how children’s SEC develops in response to school-based SEL.

Authors

• Juyeon Lee, The University of Hong Kong
  Presenting Author

• Valerie Shapiro, University of California, Berkeley
  Non-Presenting Author

• Jennifer L Robitaille, Aperture Education
  Non-Presenting Author

• Paul LeBuffe
  Non-Presenting Author