Methodological Guidance Paper: Conducting a High-Quality Evaluation of an SEL (Social and Emotional Learning) Intervention
Sun, April 14, 3:05 to 4:35pm, Philadelphia Marriott Downtown, Floor: Level 4, Franklin 10Abstract
This methodological guidance article discusses the elements of a high-quality social and emotional learning (SEL) evaluation in the school context. As discussed in the three prior papers, the generalizability of study effects and the consideration of for who under what conditions on what specific and meaningful outcomes is riddled with missingness, overgeneralizations, underrepresentation, and error (cite three papers here). In this guidance article, we address these gaps head on and discuss study designs, methods of data collection, analysis, and reporting that support high-quality evaluations and interpretations of effects.
Three themes underscore our guidance. First, SEL is not one program or approach, but rather refers to a constellation of skills and strategies that can range in breadth of skills taught and depth of skills covered, as well as how it is delivered and what types of outcomes are anticipated. Second, high-quality evaluations of SEL interventions require intentional decisions to beget equity. Extant evidence syntheses of SEL have demonstrated robust evidence of a field oversaturated and under-developed in its representation of effects. A final theme is that SEL must be more specific and increase rigor to meet the criticisms and support the proliferation of healthy supports for schools moving forward.
This guidance article is organized into four sections. The first section discusses study designs and key decisions to make when conducting a high-quality evaluation of an SEL intervention, including how randomized controlled trials (RCT) or quasi-experimental designs (QE) are considered the gold standard and yet for the field of SEL can be insufficient and the role of mixed methods in enhancing studies of SEL. The second section discusses best practices in data collection methods, including what types of data, aggregations, and why to ensure and support best practice in specifying program features, intervention elements, measures of fidelity of implementation, and increasing the likelihood of answering for whom do these results apply. The third section discusses contemporary methods and key considerations when analyzing the effects of an SEL intervention, including minimum power to detect effectiveness, baseline equivalence, data structures, and moderation methods critical for determining program and student differential effects. The final section discusses the presentation and interpretation of results from an SEL intervention to support field transparency and data-driven decision making, including pre registration, open science practices, reporting null effects, effect size benchmarks in the context of an SEL intervention, and translation of results to support diverse stakeholder engagement and uptake. Given the increasing investments and attention in SEL programs and policy, supporting high-quality evaluations for the future of SEL research, practice, and policy is critical.