Individual Submission Summary
Share...

Direct link:

How to read an evaluation manual

Thu, October 7, 11:30am to 1:00pm EDT (11:30am to 1:00pm EDT), 4S 2021 Virtual, 21

Abstract

This paper is a reading of a prime specimen of the how-to genre: The so-called “Handbook of evaluation questions”, issued in 1981 by the Norwegian foreign aid administration. The handbook concerned how to write a good evaluation report – and how to ideally implement a system for aid evaluation, in which all aid projects were perfectly planned and monitored in order to enable high quality evaluation. The handbook’s approach was directly inspired by a strong social science critique of foreign aid, and sought to implement social science methods for ensuring a more self-reflexive and transparent aid practice through more systematic documentation – which in turn would make evaluation possible. This paper investigates this handbook to understand its envisioned genres of evaluation and systems of evaluability. In so doing, the paper develops the concept of ‘optics of evaluation’ as a means to analyse evaluation tools, methods, practices and infrastructures (Reinertsen 2016, 2018). If we consider evaluation a ‘technology of seeing’ (Pasveer 1992), what does this technology make us see? How does the optic itself contribute to transform the object it investigates? In probing these questions, the paper extends longstanding STS interests in scientific observation (Latour & Woolgar 1986, Daston & Galiston 2007, Vertesi 2015). The paper anchors this discussion in the handbook: What is this document? How does its genre, style, and aesthetic enable a certain vision of an evaluable society? And how does the handbook expect its prescribed genre of evaluation to contribute to fulfilling this vision of evaluability?

Author