Towards Meta Principles Based Evaluation Framework For Community Sustainable Transformation Programs

• In Event: E/valuative Actions: Exploring the Doings of E/valuation in Evaluative Systems 2

Abstract

Socio-technical transformations are inherently uncertain, in possible pathways interventions can take, and their probable outcomes. Applying predetermined, output based evaluation measures can limit the learning space to narrow, quantitative insights and be restrictive in unravelling social change. Additionally, it makes evaluation “value free” and context independent. On the other hand, evaluation from a global normative values perspective may be too abstract for benchmarking or measuring change at an operational level. Functional socio-technical systems occupy this overlapping space between the two extremes, which can be characterized as the “messy” part of wicked problems, usually unaddressed in traditional evaluation frameworks. A three-tiered evaluation framework is proposed, which addresses these limitations. Empirical insights from transdisciplinary intervention experiments in a community based sustainable transformation project, “Rio Claro Sustentavel 2020” in rural Brazil, are used in this study. The top tier of this framework encapsulates a set of meta principles and operational measures of change are at the bottom. The middle tier, “governance layer,” is the dynamic evaluative space. This consists of the social institutions (including social norms and practices) and the collective social learning processes, encompassing relevant stakeholders in a socio-technical change process. It underscores a methodological need, with a variety of modes and media, that include narrative, story, metaphor, and imagery, to engage with the often contested, dynamic and uncertain trajectories of transformations. This research demonstrates how individual and group behavior, as well as social learning and memory can be useful methods to measure socio-technical shifts.

Authors

• Saurabh Biswas, Arizona State University

• Rimjhim Aggarwal, Arizona State University