A Brief History of Program Evaluation: Applications of Contemporary Motivation Research

• In Event: Practice-Relevant Motivation Research 2.0: Building a New Approach

Abstract

In this presentation, we draw a parallel between the evolution of program evaluation and motivation research in order to identify a roadmap for applied motivation research. Initially, it is difficult to see the overlap between the application-focused, context-based work conducted by evaluators and the theory-building, generalizability-focused research conducted by motivation researchers. However, there are insights in the history of evaluation research that mirror the contemporary status of motivation research in education. As has been the case in evaluation research, scholars question the real-world utility of motivation research for stakeholders (Kaplan et al., 2012).
In reference to the state of program evaluation in the 1960s and 1970s, Wholey (1970) wrote that “the recent literature is unanimous in announcing the general failure of evaluation to affect decision making in a significant way” (p. 46). As stated aptly by Berk and Rossi (1999), the notion of evaluation “derives from the commonsense idea that social programs should show demonstrable benefits” (p. 1). Program evaluation is therefore a practical endeavor, and its purpose is to judge the success of a specific project or program through systematic investigation (Royce, Thyer, & Padgett, 2010). Despite the importance of evaluation work for demonstrating the impact of program efforts, evaluation studies were not being widely conducted in the 1960s and 1970s—particularly not in a manner that produce useful information for stakeholders (Patton, 2008).
In the 1980s, an idea took hold within evaluation which may now prove valuable for motivation researchers: use or utilization (Patton, 2000). When the use of evaluation findings becomes a standard by which to judge evaluations, it is no longer sufficient for evaluators to conduct evaluations that are simply methodologically or theoretically sound—they must make sure that their evaluation efforts are useful to stakeholders (Patton, 2000). Research on how evaluations are used by stakeholders has demonstrated that use is more likely to take place when stakeholders are engaged in the evaluation process (Cousins & Leithwood, 1986). For this reason, utilization-focused evaluation involves upfront and continual negotiation and collaboration with stakeholders and intended users to make decisions on evaluation purpose, questions, data collection, analysis, and reporting format (Patton, 1988, 2000). Identifying how the evaluation findings will be used is an iterative process that the evaluator undertakes with the stakeholders that does not end until use has taken place (Patton, 1988, 2000).
Researchers continue to stress the importance of expanding motivation research to include greater conceptual and methodological detail in application-focused studies and classroom-based approaches (Kaplan et al., 2013). The history of utilization-focused evaluation offers a cross-disciplinary lens and new insights as researchers critically discuss ways of strengthening ways of conducting applied motivation research.

Author

• Chloe Ruff, Gettysburg College