Reviewer Stage

• In Event: Open Science 201: Research Life Cycle From Various Stakeholder Perspectives

Abstract

This panelist will talk from the perspective of a journal reviewer, discussing how open science practices are relevant to reviewing journal article submissions. Particular emphasis will be placed on discussing the pros and cons of open peer review, how to request open science practices through review, and how to promote TOP guidelines within the reviewer context.

Attendees will leave the session ready to implement strategies of open science and open peer review using a critical eye to improve transparency and practices through peer review.

A. Relevant Terms
We will define open peer review through the lens of open signing, open reports, and/or open participation in reviews, and TOP guidelines.

B. Background Information
Peer review is often shrouded in secrecy, but is the newest avenue for the open science community to tackle transparency (Wolfram et al., 2020). Open peer review can consist of signed reviews (i.e., the reviewer gives their name), open reports (i.e., the publication of the peer review with the published article), or open participation (i.e., open commentary on published articles through services like hypothes.is). Open peer review has increased in use since the early 2000s and appears most commonly in medical disciplines within a few publishers. In this discussion, I will provide examples of practices implemented by Meta-Psychology, an open access journal that requires open peer review. Further, the demands placed on reviewers is often high, which can be lessened by providing a TOP guidelines friendly set of templates for reviewers to encourage open science practices through review.

C. Expected Challenges and Barriers
Open peer review practices can be daunting to researchers, especially early career academics who do not want to “make waves” in their scientific community.
Requesting authors to follow TOP guidelines may be difficult without journal support.
Not all reviewers will feel comfortable checking all parts of the manuscript, and authors may not respond when asked (i.e., data, scripts, etc.).

D. Tips of Trade
Journals should give TOP structured templates to reviewers to help guide reviews.
Start small by encouraging, but not requiring open review, and the normalization of this process will promote its use.
Incentivize reviewing. Peer review can be published alongside the article with a citable DOI to encourage open practices and reward effort that is currently underappreciated.

Wolfram, D., Wang, P., Hembree, A. et al. Open peer review: promoting transparency in open science. Scientometrics 125, 1033–1051 (2020). https://doi.org/10.1007/s11192-020-03488-4

Author

• Erin M. Buchanan, Harrisburg University of Science and Technology