Sustaining fidelity to an evidence-based intervention in a community-based organization

• In Event: 1-159 - Leveraging a major component of Attachment and Biobehavioral Catch-up to ensure successful community dissemination

Integrative Statement

Delivering evidence-based interventions with high fidelity is a critical component of successful dissemination (Durlak & DuPre, 2008; Dusenbury et al., 2003). However, for sites that are disseminating evidence-based interventions in the community, sustaining high fidelity is a feat, with few interventions reporting or monitoring fidelity (Durlak & DuPre, 2008). To achieve sustainability, Hogue et al. (2013) advocate for moving from “purveyor-driven dissemination methods” to localized quality assurance with community organizations autonomously monitoring and supervising fidelity. Attachment and Biobehavioral Catch-up (ABC) is a 10-session evidence-based parenting program, delivered in the home by parent coaches, that aims to enhance parental sensitivity. Fidelity to ABC is defined as the frequency, accuracy, and quality of in-the-moment comments that coaches make about parent-child interactions. Fidelity to ABC is monitored by the ABC team throughout a full training year, with certification determined at the end of training. Post-certification, coaches work independently of the model developers, with fidelity procedures determined by local agencies. Leveraging ongoing dissemination efforts at a non-profit agency in a large city, the current study examined trajectories of fidelity to ABC from pre-certification (when fidelity monitoring and supervision was developer-driven) to post-certification (when monitoring and supervision shifted to localized quality assurance). This presentation aims to describe the iterative process by which the local agency developed their fidelity monitoring and supervision structure, and present empirical findings regarding trajectories of fidelity across varied levels of support. The ABC team continued fidelity monitoring after the certification period so that the local agency could establish a data-driven fidelity monitoring and supervision structure with three levels of tiered support. At Level 1, all coaches received reports summarizing their fidelity statistics on a regular basis. At Level 2, implemented 6 months post-certification, all coaches also received monthly fidelity-focused supervision provided at the local agency that included a review of fidelity metrics from a session coded by a local supervisor and the coach, practice-oriented supervision (e.g., video review, role-playing), and setting goals for fidelity performance. At Level 3, implemented with coaches who consistently fell below fidelity criteria, a local supervisor provided fidelity-focused supervision on a weekly basis (Figure 1). Preliminary analyses are available for five coaches. Hierarchical linear modeling was used to model fidelity changes, accounting for repeated measurements of fidelity assessments nested within coaches. From the period of pre-certification to post-certification (before the local agency initiated fidelity support), on average, there was a significant decline in rate of commenting, β10 = -.47, p = .028. After fidelity support was offered by the local agency, on average, rate of commenting increased, β10 = .37, p = .144. Although this slope was non-significant, performance of all five coaches exceeded fidelity criteria (Figure 2). Continued data-collection efforts will assess the effectiveness of the fidelity support model instituted by the local agency. This study highlights the need for fidelity monitoring and support, as high fidelity ultimately leads to improvements in parental behavior and child outcomes. Assessing this fidelity monitoring model will also inform the development of supports at other ABC dissemination sites.

Authors

• Fabianne Blake, University of Delaware
  Presenting Author

• Kristin Bernard, Stony Brook University
  Non-Presenting Author

• Allison Dash, Power of Two
  Non-Presenting Author

• Stevie S. Schein, University of Delaware
  Non-Presenting Author

• Amanda H. Costello, University of Delaware
  Non-Presenting Author

• Caroline K.P. Roben, University of Delaware
  Non-Presenting Author

• Mary Dozier, University of Delaware
  Non-Presenting Author