Looking Inside and Across Frameworks: Findings from Preliminary Analysis of the Taxonomy Project

• In Event: 3-189 - Using Indicators of Social-Emotional Development to Inform Practice

Integrative Statement

Prevention science is grounded in the assumption that research drives what we do in intervention contexts, both in terms of what we promote in children (the specific skills or outcomes) and how we measure it (how we assess children’s skills or evaluate the impact of interventions). Importantly, before we can begin to accurately measure impact and outcomes in this area, we must be able to be clear about the skills we hope to target and assess. However, discussion of the social, emotional, and “non-academic” domain is beset by dilemmas about how best to measure and promote skills in this area – in part because the domain is structured around a large number of organizing systems, or frameworks, that often organize and describe skills using different or even sometimes contradictory language (Reeves & Venator, 2014). Without a way to make sense of the words we use and the definitions we ascribe to them, we risk misaligning skills, strategies, and measures in ways that do not build and assess the skill or achieve the outcomes we intend. To respond to this challenge, we developed a rigorous coding system that when applied to frameworks in the field, can be used to identify related skills across them, thus linking terms by how they are defined rather than by what they are called.
This paper summarizes the methods we used to address the following questions: (1) how closely related are the discrete skills in each framework based on their definitions; (2) based on this information, and looking across frameworks, how much agreement is there in the type of skills they include; and (3) what are the major clusters of skills that appear when we strip away names and focus on definitions? Our methods included using a common coding system to look across 16 evidence-based frameworks to record when the definitions/descriptions they provide for skills target 150+ common SEL skills (e.g., “identifies emotions in others”) across 22 sub-domains (e.g., empathy, conflict resolution, etc.) and 6 domains (e.g., emotion, interpersonal, etc.) and devising an algorithm to calculate the similarity between two terms based on how many overlapping codes they received. The frameworks include a representative sample of domains, age groups, and settings in the U.S. and internationally.
Findings include a quantitative analysis of how frequently skills with the same name have different definitions, how often skills with different names have similar definitions, and how terms are clustered based on the coding of their definitions. These findings highlight areas of commonality across frameworks and provide educators, researchers, funders, and policymakers with a way to understand and connect the skills they seek to measure and promote in children. We also discuss the future potential of our coding system and methods to build greater clarity and precision in the field when used to link skills and frameworks to strategies and measures, with the goal of creating a research-to-practice cycle that accurately promotes and measures social, emotional, and other “non-academic” skills.

Authors

• Bryan Nelson, Harvard Graduate School of Education
  Presenting Author

• Katharine Elizabeth Brush, Harvard Graduate School of Education
  Non-Presenting Author

• Rebecca Bailey, Harvard Graduate School of Education
  Non-Presenting Author

• Stephanie Jones, Harvard Graduate School of Education
  Non-Presenting Author