1-184 - Measuring Social Emotional Competence in School Settings: Exploring Assessment using Multiple Raters and Multiple Methods

Session Type: Paper Symposium

Integrative Statement

Social emotional learning (SEL) programs can prevent negative outcomes (i.e., aggression, anxiety, delinquency) and promote positive youth development (Durlak et al., 2011). There is a need for psychometrically sound assessment tools to facilitate the implementation and evaluation of SEL programs in schools. All three papers use the 72-item Devereux Student Strengths Assessment (DESSA; LeBuffe, Shapiro, & Naglieri, 2009) or the 8-item DESSA-Mini (Naglieri, LeBuffe, & Shapiro, 2009) to explore measurement questions related to SEL assessment.

The first paper examines rater bias through behavior rating scales. This study found that only 16% of variance in assessment scores was attributable to raters. Collectively, teachers’ perception of barriers to student learning, report of their “typical” student’s Social Emotional Competence, anticipation of SEL program implementation challenges, and intention to fully implement the SEL program explained nearly 6% of the systematic bias in scores.

The second paper explores convergence between youth self-report and adult informant ratings. The distribution of student self-report and adult rating scores of students’ Social Emotional Competence were not significantly different, although correlations were low. Students and staff indicated a “fair” degree of agreement, consistently classifying students who had a “Need for Instruction,” 79% of the time.

The third paper compares scores generated through a direct assessment of Social Emotional Comprehension to scores generated through an indirect assessment of Social Emotional Competence. Findings indicate significant correlations between methods.

Together, these papers will stimulate a discussion about an ongoing research and practice agenda for multi-rater and multi-modal assessment of SEL in schools.

Sub Unit

• Prevention and Interventions

Chair

• Valerie Shapiro, UC Berkeley

Discussant

• Clark McKown, Rush University Medical Center

Individual Presentations

• Role of Rater Characteristics in Understanding Rater Bias in Social Emotional Competence Assessment - Presenting Author: B.K. Elizabeth Kim, University of Southern California; Valerie Shapiro, UC Berkeley; Sarah Accomazzo, University of California, Berkeley; Joe Roscoe, University of California, Berkeley

• Comparing Student Self-Report and Staff Ratings on a Strengths-Based Assessment of Social Emotional Competence - Presenting Author: Sarah Accomazzo, University of California, Berkeley; Valerie Shapiro, UC Berkeley; Sophie Shang, University of California, Berkeley

• Combining Direct Assessment and Behavior Rating Scale Approaches: Examining the Relationship between VESIP and DESSA Scores - Presenting Author: Nicole M Russo-Ponsaran, Rush University Medical Center; Valerie Shapiro, UC Berkeley; Sarah Accomazzo, University of California, Berkeley; B.K. Elizabeth Kim, University of Southern California