Hidden Impact: Taking Relationships Seriously When Evaluating Nonprofit Social Change

• In Event: Mismeasuring Nonprofit impact: What Goes Wrong and How to Do It Better

Abstract

The second book project, Hidden Impact: Taking Relationships Seriously When Evaluating Social Impact, shows that evaluations of program interventions neglect the central role of staff-participant relationships in supporting positive change.  Although relationship techniques and strategies are sometimes the core feature of an intervention and thus part of an evaluation, Hidden Impact shows that these relationships and their effects have been under-conceptualized in program evaluation.  This presentation will focus on the historical development of the program evaluation field to explain why relationships and their effects have been neglected in program evaluation.

 

After summarizing evidence that relationships are significant for supporting positive change, the presentation examines why relationships have been under-specified in program evaluation, even when early evaluation experts recognized that interventions unfolded in social settings that affect implementation processes and outcomes. I show how social scientists’ initial hopes that evaluation would make an indispensable contribution to social problem solving gave way to the realization that they had been naïve. Not only were they finding that these social programs had little to no effect on important social problems, but their notions of how evaluation would feed into a rational decision-making process was upended as they found themselves mired in a political environment that shaped not only which evaluation questions were asked but whether and how evaluation results were used. I describe how these early experiences shaped the program evaluation field for decades to come, with consequences for how relationships were treated as the field developed. For example, initially relationships were seen as part of the context that can support or stymie implementation of an intervention. Later these staff-participant relationships were seen as a place where evaluations could intervene—to realize certain values such as equity and justice–by including those with the least power (the program participants) in the evaluation and elevating their voice. Today, several evaluation experts have made these relationships more central in their evaluation approaches. But I posit that because relationships have been ‘retrofitted’ onto a basic evaluation model, the significance of relationships remains underappreciated, and programs or interventions are the central ‘actors’ of social change. I conclude by discussing the implications for managers in human services.  

Author

• Lehn Marie Benjamin, Indiana University
  Presenting Author