Examining Cross-cultural evaluation practices: Leveraging co-evaluation methodologies to merge knowledge systems and research practices

• In Event: A Multinational e-Learning Partnership: Coordinating New Opportunities for Student Learning Across 10 Higher Education Institutions

Proposal

Session Abstract
Cross-cultural collaborative evaluations can significantly transform institutions and stakeholders, ensuring culturally relevant evaluation design and methods inform evaluation design. In this paper, we share the experiences of a US public university and a private university in Africa on the process of 1) forming a cross-cultural evaluation team, 2) co-developing evaluation frameworks, 3) co-designing and piloting instruments, and 4) conducting data collection and analysis for e-Learning Initiative implemented across ten higher education institutions, mainly in Africa. We highlight attempts to ‘decolonize’ evaluation by working hand in hand with key stakeholders, which resulted in transformative evaluation designs and enhanced data collection and analysis. As we explore some promising practices and challenges in working virtually amid the COVID-19 pandemic, we also share self-reflective moments that challenged us to consider equity and inclusiveness in working with different knowledge systems, virtual working environments, and research practices.

Relevance Statement
In recent years, collaborative approaches to evaluation are the most often used for cross-cultural inquiry associated with interventions for developing countries and/or low-resource settings (Chouinard & Cousins, 2007; Hoare, Levy, & Robinson, 1993). This is due to the critiques of the overrepresentation of donor priorities in programming and evaluation, as opposed to the emphasis on the needs and interests of the local communities (Carden, 2010; Hay, 2010). Instead, cross-cultural, collaborative evaluation is considered an alternative way to replace the traditional approaches that “privilege rigor over the importance of contexts” and may even be considered “oppressive concerning the learning function of evaluation” (Cousin et al., 2013, p.7).
Collaborative evaluation allows stakeholders (e.g., implementing partners, participating institutions) who have participated in the evaluation process to develop a stronger sense of ownership. When implemented appropriately, collaborative evaluation becomes transformative in the interests of social inclusion and justice and the amelioration of social inequity. Moreover, it is only through such equitable, collaborative interactions that the evaluation team could gain a deeper understanding of the complex program phenomena and greater (external) validity (Cousins et al., 2016).
Challenges in cross-cultural collaborative evaluation could be challenging as it is sometimes considered more time-consuming (e.g., not enough time to approach and gather views from all), and some may maintain a hard stance on the entire process (Pollitt, 1999). Moreover, even with a carefully crafted evaluation plan, it could be difficult to accurately accommodate the perspectives of all stakeholders and keep them actively engaged throughout the process (Rodrıguez-Campos, 2012). Hence, it is crucial that collaborative evaluators understand and assess potential barriers to collaboration and create opportunities to overcome them, which could be done through reflective and learning sessions (O’Sullivan, 2012).
In this paper, we seek to share our promising practices and lessons learned from the cross-cultural, collaborative evaluation process. Our presentation will include how our research team addressed issues of equity and inclusion among ourselves and in the evaluation design. This paper aims to engage the audience in a critical dialogue about experiences, successes, and challenges around the role of collaborative research in decolonizing evaluation.

Author

• Ann Nielsen, Arizona State University