Quality of Delivery Monitoring and Supporting Non-Specialists to Deliver Evidence-Based Interventions

• In Event: Monitoring, evaluation, research and learning in ECD

Proposal

There are significant mental health treatment gaps facing children and youth in low- and middle-income countries (LMICs). In recent years, researchers and practitioners have addressed this gap by task-shifting evidence-based mental health and psychosocial support (MHPSS) and early child development (ECD) interventions to non-specialist community providers, such as community health workers or other non-specialist providers. Task-shifted interventions have demonstrated effectiveness (improvements in MHPSS or ECD outcomes of intervention participants); however, quality of delivery (fidelity and competence) has rarely been examined as a factor associated with effectiveness despite the fact that implementation science models suggest a causal link between quality of delivery and intervention effectiveness.

This applied research study examines the quality of delivery of non-specialists delivering an evidence-based early childhood development (ECD) and family violence prevention project, known as Sugira Muryango, which is implemented in three rural districts of Rwanda. Sugira Muryango is designed around the Nurturing Care Framework, and equips caregivers to provide holistic support for their children, in addition to caring for their own mental health and interpersonal relationships. The structure of Sugira Muryango consists of 12 modules that are delivered to families in a home-visiting format, in addition to two follow-up/booster sessions.

Currently, Sugira Muryango is being expanded and implemented through the Promoting Lasting Anthropometric Change and Young Children’s Development (PLAY) Collaborative, which scales up Sugira Muryango to 10,000 households in Ngoma, Nyanza, and Rubavu districts, including nationally-run ECD centers throughout the country. The program has strong ties to the Rwanda National Government and their social protection and policy goals. A pre-existing government volunteer child protection workforce, the “friends of the family” or Inshuti z’Umuryango (IZUs), serve as the non-specialists delivering the intervention.

In this study, we first uses a latent class regression analysis to examine which characteristics of non-specialist providers (such as supervision experiences and demographic factors) are associated with greater quality of delivery. Second, we examine the association of non-specialist quality of delivery scores with intervention outcomes in Sugira Muryango. Multi-level modeling is used to examine summative quality of delivery score (fidelity + competence), as the key predictor of changes in intervention outcomes from baseline to endline (N = 222 households, N = 213 children). Primary intervention outcomes that we use in this study are child discipline practices and responsive caregiving of children. Child discipline practices include items pertaining to explaining why the behavior was wrong, physical punishment, or psychological aggression, and is measured via the UNICEF Multiple Indicator Cluster Survey. Responsive caregiving is measured via the Home Observation for Measurement of the Environment (HOME) Inventory, and includes six caregiving behaviors: use of learning materials, variety, responsivity, acceptance, organization, and involvement.

Findings demonstrate that non-specialists receiving more regular monitoring, supervision, and mentorship have higher quality of delivery throughout the intervention and improve their quality of delivery more quickly. Furthermore, findings demonstrate that quality of delivery is associated with greater frequencies of non-violent child discipline practices, and more responsive caregiving techniques. As task-shifting MHPSS and ECD interventions to non-specialists continues to be a widely used and effective strategy for addressing the treatment gap in LMICs, this project contributes important insights into the factors associated with quality of delivery and how quality of delivery is associated with mental health improvements. As researchers and practitioners continue to report on the effectiveness of MHPSS interventions and move towards mechanistic work, measuring and including quality of delivery as a variable in modeling, as validated in this study, may present a fuller picture of effectiveness, and allow us to account for more variation in data. However, this research has also revealed a number of ways that quality of delivery monitoring could have been improved throughout the PLAY Collaborative program, which includes conceptual distinction of fidelity vs. competence and checklist items that operationalize these concepts correctly, the use of an improved Likert scale for rating items in the checklist, and better practices for training, supervising, and mentoring non-specialists as they deliver Sugira Muryango to families and in ECD centers. For example, the use of a digital dashboard or other training materials that IZUs can access on their own time can enable IZUs to learn the content of Sugira Muryango and practice competencies such as active listening, problem-solving, and activating risk of harm and safety plans.

Research findings can also inform strategies for training and supporting non-specialists, during and after implementation, and ultimately, strategies for scaling and sustaining MHPSS and ECD interventions for children and families. Many agencies and individuals attending CIES use non-specialist providers in communities to deliver a range of evidence-based interventions and programming, and this applied research study can generate conversations regarding lessons learned, available resources, and best practices for ensuring that community members have the skills, support, and assets available to assume more responsibility and leadership over programming being done in their own communities.

Authors

• Laura Bond, Boston College

• Theresa S. Betancourt, Boston College