The Association Between Youth Perceptions and Fidelity Ratings of Residential Evidence-Based Practices

• In Event: Evaluating the Promise of Evidence-Based Practice in Juvenile Justice Settings: Short and Long Term Feasibility

Abstract

Much progress has been made in the dissemination of evidence-based practices and programs (EBPs) in the juvenile justice system. Successful implementation of EBPs, however, requires robust quality assurance mechanisms to achieve desired outcomes. While many community-based EBPs have already developed strong fidelity monitoring systems, less progress has been made for institutional EBPs that rely on milieu-based treatment models. Commentators propose that such quality assurance systems should include standardized ratings of treatment milieu adherence completed by trained staff observers as well as similar, but developmentally modified, adherence ratings completed by youth themselves. It remains uncertain, however, whether staff ratings or youth ratings, or a combination of the two, provide the most valid overall assessment of model fidelity and the quality of the treatment milieu. This study addresses this question using data from staff and youth ratings of a Dialectical Behavior Therapy model implemented in Washington State juvenile residential settings. Preliminary results suggest that staff ratings are a reliable measure of model fidelity. The validity of these ratings, as well as the reliability and validity of youth ratings are currently being analyzed. Study findings will inform best practices for evaluating and monitoring treatment fidelity for milieu-based EBPs in juvenile justice more generally.

Authors

• Asia Bishop, University of Washington

• Sarah Walker, University of Washington

• Terry Lee, University of Washington

• Henry Schmidt, Behavioral Affiliates, Inc.

• Sarah Veele, Washington State Rehabilitation Administration

• Debbie Lyne, Washington State Rehabilitation Administration