First, You Need to Understand Mississippi: Accepting Responsibility for Understanding Your Context

• In Event: Improving Equity Through Culturally Responsive Evaluation Practice

Abstract

Objective: The purpose of this presentation is to describe one Mississippi-based evaluation center’s efforts to take responsibility for understanding the context in which they work. This presentation will share the process the evaluation center undertook to (1) build a deeper understanding of the racial, cultural and contextual history of Mississippi and (2) develop a framework for conducting equity-focused evaluation within that context. The presentation will also share the final framework and the challenges the team experienced along the way.

Theoretical perspectives: Evaluation theorists and practitioners regularly argue that context matters in evaluation. Evaluator competency lists from around the world dedicate entire competency domains to understanding and responding to context. The American Evaluation Association’s set of Evaluator Competencies charges evaluators with responsibility for “understanding the unique circumstances, multiple perspectives, and changing settings of evaluations and their users/stakeholders.” Similar domains are found in competency lists in Australia, Canada and New Zealand. One rationale for this focus on context is the argument that we can better facilitate the use of research and evaluation when our work is aligned to—and appropriate for—the context in which it is being conducted (e.g. Chen, 2014; Shadish, Cook & Leviton, 1991). Other literature argues that the validity of such studies also rests on evaluators having an extensive (and accurate) understanding of the context in which the research occurs (i.e., cultural validity; Kirkhart, 2010). This is echoed in wider literature that suggests uncovering “truth” requires “deep and valid knowledge of context”—including “the origins and repercussions of historic or existing laws or systems” (Bledsoe, 2014).

Modes of inquiry: This center used an action research model to develop their situationally-specific framework for conducting equity-focused evaluation in Mississippi. By partnering with a non-profit organization that has deep ties to Mississippi and a long history of work in racial reconciliation and equity-focused civic dialogue, this center collected a wide variety of data about structural and systemic inequities in Mississippi, in addition to data on the way the center conducts its own work.

Evidence, objects, materials; results: Through a series of collaborative workshops and critical feedback, this center developed a conceptual framework for conducting equity-focused evaluation in Mississippi, critiqued its work against that framework and agreed upon future-oriented strategies for improvement. The non-profit, serving as a critical friend, reviewed and helped refine this framework, applying the lens of their work related to racial reconciliation in Mississippi.

Significance: The process—and product—we share will be of value to other educational evaluators who seek to take responsibility for understanding the context in which they work.

Bledsoe, K. L. (2014). Truth, Beauty, and Justice: Conceptualizing House's Framework for Evaluation in Community‐Based Settings. New Directions for Evaluation, 2014(142), 71-82.

Chen, H. T. (2014). Practical program evaluation: Theory-driven evaluation and the integrated evaluation perspective. Sage Publications.

Kirkhart, K. E. (2010). Eyes on the prize: Multicultural validity and evaluation theory. American Journal of Evaluation, 31(3), 400-413.

Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Sage.

Author

• Sarah Mason, The University of Mississippi