Exploring Community-Driven and Culturally Responsive Approaches to Evaluation Capacity Building

• In Event: Building Evaluation Capacity in the Early Childhood Field to Enhance System Effectiveness

Abstract

Historically, evaluation has played a role in the marginalization of specific communities. To counter top-down evaluation paradigms and ensure that the demand for and production of evaluative information reflects the plurality of culture and is actionable at the community level, there is a need to transform evaluation approaches, methods, and training. Including concepts of cultural responsiveness in evaluation, which involves actively recognizing, appreciating and incorporating culturally related contextual factors into practice (SenGupta et al., 2004), reduces inaccuracies which may occur when evaluators presume an understanding of culture (Askew et al., 2012). Further, cultural responsiveness enhances the ability to identify appropriate considerations of cultural context in various aspects of the evaluation process (Askew et al., 2012). To investigate how evaluation can become a community-driven and culturally responsive mechanism for improving practice, programs, and policy, the Evaluation Capacity Network is conducting a series of scoping reviews, grey literature scans, and key informant interviews.

The purpose of the reviews is to explore how evaluators currently understand and practice community-driven and culturally responsive evaluation, and identify promising practices, frameworks, and approaches. Specifically, a total of three scoping reviews are being conducted pertaining to: i) community-driven evaluation; ii) culturally responsive evaluation; and iii) evaluation in the early childhood development field, which intersects both community-driven and culturally responsive scoping queries. All three reviews are guided by the Arksey & O’Malley framework (2005). To date, 141 articles have been included in the culturally responsive evaluation review, and 22 in the ECD review after initial title/abstract screening and full-text review. The community-driven evaluation review is still in the early stages of title/abstract screening for 600 identified articles. Preliminary review findings demonstrate a significant variation in how collaborative approaches are practiced, including the extent to which stakeholders are engaged and the types of roles that stakeholders play in the evaluation process. One identified challenge is the lack of evaluators or researchers who have sufficient knowledge of the community and its culture, language, as well as the power imbalances and organizational constraints to conduct culturally responsive evaluations. Recognizing the limitations of relying solely on a peer-reviewed evidence base to deeply understand community-driven and culturally responsive approaches, key informants across multiple sectors and disciplines will be further engaged. ‘Engagement toolkits’ that draw on multiple engagement processes and tools (e.g., stimulus papers, interviews, community dialogues, engagement curriculum) will be developed to guide dialogues. Parallel to these activities, there are also several ongoing community-based research and evaluation projects that will serve as case studies for testing numerous approaches to understand how capacity can be built to improve the creation and use of culturally responsive and contextualized evidence that informs practice, programs, and policy in early childhood.

In this presentation, we will: i) summarize preliminary findings from the scoping reviews; ii) outline the process of developing engagement toolkits; and iii) highlight emerging themes from key informant interviews.

Authors

• Rebecca Jayne Gokiert, University of Alberta

• Pieter de Vos, University of Alberta

• Cheryl-Anne Nadine Poth, University of Alberta

• Michelle J Searle, Queens University Kingston Ontario

• Shelly Jun, University of Alberta

• Giulia Puinean, University of Alberta

• Mischa Taylor, University of Alberta

• Christopher Mushquash, Lakehead University

• Shevaun Nadine, Lakehead University

• Mary Ellen Hill, Lakehead University

• Abbey Radford, Lakehead University