Mismeasuring Nonprofit impact: What Goes Wrong and How to Do It Better

Session Submission Type: Panel

Abstract

The demand for evidence of nonprofit social impact is at present a taken-for-granted assumption for the public policymakers and public managers who shape and oversee our public-private system of social service provision. Evaluation metrics are required elements of the government contracts that support many of the nonprofits serving individuals, families, and communities across the nation. The practice of nonprofit evaluation, however, is laden with challenges. Drawing on three book projects, this panel examines what goes wrong in the status quo of nonprofit evaluation and how we might do better. The first project examines the consequences of recent public policies that favor randomized controlled trials (RCTs) to evaluate nonprofit programs. It argues that RCTs are a poor match for the nonprofit context, as they undermine these organizations’ core strengths of responsiveness and innovation. The second project identifies a central reason for the mismatch between standard evaluation approaches for nonprofit programs and the possibilities for social change strategies: the under-theorization of how much the relationships between nonprofit participants and organizational staff contributes to nonprofit social impact. Drawing on theory from sociology and social psychology, it demonstrates the importance of reshaping evaluation to account for nonprofit-participant relations if the impact of nonprofits is to be accurately captured. The third project moves the conversation further, showing how funders and community leaders struggle and sometimes succeed in reconciling standardized metrics with local adaptation and responsiveness. 

Policy Area

• Public and Nonprofit Management and Finance

Chair

• David F Suarez, University of Washington

Discussant

• Mary Kay Gugerty, University of Washington

Organizer

• Nicole Marwell, University of Chicago

Individual Presentations

• Mismeasuring Impact: How Randomized Controlled Trials Threaten the Nonprofit Sector - Presenting Author: Nicole Marwell, University of Chicago

• Hidden Impact: Taking Relationships Seriously When Evaluating Nonprofit Social Change - Presenting Author: Lehn Marie Benjamin, Indiana University

• Democracy’s Hidden Heroes - Presenting Author: David C Campbell, American Society for Public Administration

• Collective Capacity Development for a Sustainable Nonprofit Sector: The Case of the Dream City - Presenting Author: Emily Nwakpuda, University of Texas at Arlington