Fidelity predicts ABC intervention effectiveness in nation-wide community implementation

• In Event: 1-159 - Leveraging a major component of Attachment and Biobehavioral Catch-up to ensure successful community dissemination

Integrative Statement

Monitoring fidelity to evidence-based interventions is a critical component of successful dissemination (Durlak & DuPre, 2008); however, it is also challenging for many community-based providers to clearly delineate key fidelity criteria and measure them reliably in community settings (Schoenwald et al., 2011). Because of this, community implementation of interventions often loses effectiveness when compared to randomized clinical trials that are implemented in laboratories. Attachment and Biobehavioral Catch-Up (ABC) is currently being disseminated in several community settings across the United States. Fidelity to ABC is based on in vivo feedback (via in the moment comments), considered the “core component” of ABC, to improve caregivers’ sensitive and synchronous behaviors toward their infants. In community implementation, during their ABC training year, parent coaches receive weekly feedback about their fidelity via samples of moment-by-moment behavior coding of their home visiting sessions. This feedback includes the number of comments that parent coaches make per minute, as well as the percentage of their comments that conveyed accurate and ABC-relevant information to the caregivers. Additionally, before and after ABC is implemented, parent coaches administer a 9-minute play assessment, based on the NICHD Observational Recording of the Caregiving Environment (ORCE; NICHD ECCRN, 1996) in order to assess caregiver sensitivity. In this assessment, caregivers are asked to interact with the target child and a standardized set of toys. Research coders unaware of implementation site and assessment timing at [University omitted for review] score caregivers for responsiveness and intrusiveness on separate 5-point scales. The current study represents an ABC community implementation sample of 65 parent coaches across 36 diverse organizations, with outcome data from 456 caregiver-child dyads. Coach fidelity information was averaged across the ABC training year so that each individual had 2 average fidelity scores: one that represented their accuracy in representing ABC targets (percentage on-target), and one that represented the number of in the moment comments made per minute. To create our outcome variable of change in overall caregiver sensitivity, we reverse-scored intrusiveness and averaged it with responsiveness. Overall caregiver sensitivity change was then averaged across multiple cases within clinicians, to create a single “effectiveness” score for each clinician that describes the amount of change in caregiver sensitivity from pre-ABC to post-ABC. Results indicated that the number of ITM comments per minute predicted overall caregiver sensitivity change, F (1, 63) = 5.702, p <.05, R^2 = .083 (see Figure 1). The percentage of on-target comments likewise predicted overall caregiver sensitivity change, F (1, 63) = 6.134, p <.05, R^2 = .089 (see Figure 2). This indicates that more frequent, high-quality comments by the parent coach lead to greater changes in caregiver sensitivity from pre- to post-ABC intervention. This study suggests that better fidelity to the ABC intervention model predicts bigger improvements in caregiver sensitivity. This type of analysis is typically only conducted in efficacy trials in a laboratory, rather than through community implementation. These findings indicate that high model fidelity can be used to maintain high levels of intervention effectiveness when ABC is implemented in community settings.

Authors

• Stevie S. Schein, University of Delaware
  Presenting Author

• Amanda H. Costello, University of Delaware
  Non-Presenting Author

• Caroline K.P. Roben, University of Delaware
  Non-Presenting Author

• Amanda M. Flagg, University of Delaware
  Non-Presenting Author

• Mary Dozier, University of Delaware
  Non-Presenting Author