Monitoring Implementation of the Promoting Alternative Thinking Strategies (PATHS) Curriculum: Lessons From a District-Wide Social Emotional Learning Initiative

• In Event: Building Infrastructure for Social Emotional Learning in Routine Practice: Assessing Student Competence and Instruction Fidelity

Abstract

Objective: Monitoring implementation fidelity is essential to implementation success and intervention effectiveness. Ongoing monitoring, however, is challenging. The Promoting Alternative Thinking Strategies (PATHS) Curriculum is a well-established SEL program (Greenberg, Kusche, Cook, & Quamma, 1995; Kam, Greenberg, & Kusche, 2004). Although findings suggest higher implementation quality is critical to achieving youth outcomes (Kam, Greenberg, & Walls, 2003), research-informed guidance is unavailable to suggest how implementation quality should be monitored. Current guidelines recommend a PATHS technical assistance provider (TA) observe 20% of PATHS lessons. However, monitoring 8-10 lessons in every classroom is challenging in routine practice, particularly in a district-wide implementation. This paper seeks to understand: 1) which elements of initial observations predict completion rates of the recommended number of observations; 2) which elements of observations predict concurrent impressions of overall implementation quality; and 3) which elements of initial observations predict sustained implementation quality.

Methods: Data come from the first year of a district-wide implementation of PATHS. TAs conducted observations in 170 classrooms across 15 schools, attempting to observe each classroom 8 times. TAs observed many aspects of implementation, including teacher characteristics (e.g., teacher is committed to implementation), adherence (e.g., teacher uses PATHS techniques), participant responsiveness (e.g., students enjoy PATHS activities), and Overall Implementation Quality (an independent point-in-time rating). Using multilevel modeling to account for clustering within schools, we examined the unique effect of each observation element on completion rates and impressions of Overall Implementation Quality within and across time. Differences between TAs were examined.

Results:

Completion Rates - Observation completion rates ranged from 94% (Time 1) to 32% (Time 8). Implementation Quality was high and correlated across the school year among those observed. Initial Implementation Quality was unrelated to observation completion rates. Teachers’ initial commitment to high-level of implementation predicted completion rates.

Implementation Quality - Within each time point, teacher characteristics and participant responsiveness generally predicted Implementation Quality. Adherence, however, was only predictive for some lessons. Across time, teacher characteristics and participant responsiveness were stable, but adherence varied by lessons. Finally, initial Participant Responsiveness predicted end-of-year Sustained Implementation Quality ratings among those observed at Time 6-8.

Conclusion: Identifying early predictors of completion rates and Implementation Quality could improve sustained implementation monitoring and quality in school settings. Findings suggest initial teacher buy-in may improve implementation monitoring and quality. Early efforts to facilitate buy-in may be beneficial. Fewer observations throughout a school year may be sufficient to effectively monitor implementation, however adequate training and inter-rater reliability assessment may be critical practices to support implementation in routine practice settings.

Authors

• Jennifer Fleming Robitaille, Devereux Center For Resilient Children

• B. K. Elizabeth Kim, University of California - Berkeley

• Paul Augustine LeBuffe, Devereux Center for Resilient Children

• Valerie Shapiro, UC Berkeley