Filling In the Gaps: The Interpretation of CVs in Peer Review

• In Event: Experts of Subjective Thought: Producing Knowledge and Setting Standards - I

Abstract

In this paper we study the use of CVs for competitive funding decisions. Very little is known about how biographical information of applicants is interpreted in peer review situations. The sober administrative style typical of CVs evokes the impression of straightforwardly conveyed, objective evidence on which to base comparison. But while previous scholarship has emphasized the use of CV information for reducing uncertainty in evaluative situations, a key focus of our study is precisely on the complex interpretive politics that become possible when abstract accounts of academic career trajectories are mobilized for evaluation. More specifically, the conceptual core of our contribution is the analysis of the reflexive work through which referees make sense of biographical information. When drawing on CVs for assessment, referees typically do not just compare particular achievements of applicants, but also their own understanding of scientific practice and the assumptions that underpin CV categories. We distinguish three modalities of how such experience-based judgment can shape the comparison of applicants: Calibration, branching out, and repair. Conceptually, our analysis provides a framework for thinking about the mutually constitutive relation between CV categories and referees’ personal understanding of scientific practice. In empirical terms, it adds to existing literature on peer review by showing that even quantitative CV information can be used to perform diverse forms of comparative evaluation, and by drawing attention to the self-correcting mechanisms in the reproduction of the scientific workforce.

Authors

• Wolfgang Kaltenbrunner, Centre for Sciencs & Technology Studies (CWTS), Leiden University

• Sarah de Rijcke, Centre for Science and Technology Studies (CWTS)