Measuring Social and Emotional Learning Implementation Practices

• In Event: The Development and Initial Validation of Four Ecologically Valid, Multidimensional, and Scalable Social-Emotional Learning Assessment Tools

Abstract

This presentation focuses on an important part to understanding the effectiveness of emotional learning (SEL) approaches—the quality and degree to which educators use SEL practices with their students. We present the process of developing a school staff survey measure of SEL implementation practices as well as initial psychometric evidence. Created in collaboration with developers of the emotion-focused SEL program, RULER, the measure’s items are designed to capture SEL practices that are conceptually aligned with RULER, but are “program agnostic.” This design allows for the measure to be used in evaluation contexts for RULER participants and non-participants as it is possible that individuals with no exposure to RULER training exhibit practices that resemble SEL or emotion-focused SEL practices. Accordingly, it can be useful in experimental or quasi-experimental research designs where the aim is to compare effects of SEL practices to control groups. The measure focuses on 8 teacher-reported domains: (1) classroom culture; (2) emotion literacy; (3) emotion regulation; (4) social problem solving; (5) use of “feelings words” in instruction; (6) teacher personal use of SEL; (7) implementation support; and (8) perceptions of SEL effectiveness. For participants who were implementing RULER, they were also asked questions regarding RULER-specific implementation for the purposes of assessing validity.

Initial items were developed by the authors who then sought to identify a smaller set of items that best represented the constructs. To identify the most central items, we independently presented items to six RULER experts and developers at the Yale Center for Emotional Intelligence (YCEI). Since the pool of items we created was too large to feasibly administer to all respondents, a planned missing data design was used, which involved creating three shortened forms and that were randomly assigned to teachers within schools. During the spring of the 2018-19 school year, we collected data across two timepoints from a total of 20 schools and 668 teachers.

We used multilevel confirmatory factor analysis (MCFA) for assessing the psychometrics of each of the 8 constructs and we report on findings concerning fit indices, internal consistency, and factor loadings. To investigate validity, we used multilevel structural equation modeling (MSEM) and we report on the associations between the “RULER-agnostic” constructs and the RULER-specific implementation items as a way of assessing convergent validity. We discuss the implications of the findings and plans for additional measurement refinement as well as the application of the measure for schools using SEL programs and practices.

Authors

• Michael Strambler, Yale University

• Joanna L. Meyer, Yale University

• Almut Klara Zieher, Yale School of Medicine

• Maegan Genovese, Yale University