Can transparency undermine peer review? A computational model of scientist behavior under open peer review

• In Event: 1555 - Section on Science, Knowledge, and Technology Refereed Roundtables (1 hour)
  In Refereed Roundtable 60min: 2. "Click Publish": Peers, Review, and the Creation of Knowledge

Abstract

Transparency and accountability are keywords in corporate business, politics, and science. Since the open science movement has gained momentum in science, some journals have started to adopt forms of open peer review beyond the confidential (single or double) standard. This was to stimulate participation, promote fairness, and make reviewer contribution more visible and recognized. However, experimental trials on open peer review in scholarly journals have found contrasting evidence on the impact of these innovations on the quality of peer review. Furthermore, long-term consequences of these innovations on scientist behavior are difficult to assess with experimental procedures or empirical research. This paper aims to fill this gap by presenting an agent-based model that simulates competition and status effects between authors and referees in an idealized scenario. Results indicate that if referees are sensitive to reciprocity, possible retaliation by well-established authors and exogenous status effects, transparency can backfire on the quality of the process. This calls for further investigation on contrasting values in peer review.

Authors

• Flaminio Squazzoni, University of Milan

• Federico Bianchi, University of Brescia