Researcher: Implementation State

• In Event: Open Science 201: Research Life Cycle From Various Stakeholder Perspectives

Abstract

This panelist will talk from the perspective of a research author, discussing how open science practices are relevant to write-up and submission. Particular emphasis will be placed on discussing differences in the academic publishing experience between when using traditional vs open practices. Attendees will leave the session ready to begin implementing some open practices in their research and know how to select which practices will benefit them for which projects.
A. Relevant Terms
I will define relevant terms such as preregistration, registered reports, preprints, and open access (McBee et al., 2017; van der Zee & Reich, 2017).
B. Background Information
I will discuss my personal experiences, preregistering quantitative and qualitative studies, publishing registered reports (including those using secondary data analysis), posting preprints, and responding to reviewer comments related to open practices.
C. Expected Challenges and Barriers
1. There is a learning curve. Some practices add bureaucratic hoops and others (e.g., sharing data) can feel daunting, and policies may not always be clear.
2. Some colleagues may be hesitant. Educating collaborators about things like how preprints prevent scooping by establishing priority or that preregistration does not limit the analyses that can be used may be needed.
3. Like researchers, reviewers are people too. Preregistrations transparently communicate a priori plans and protect researchers from post hoc expectation shifting. But preregistrations do not help reviewers from being influenced by study results. Similarly, few reviewers will have a lot of experience with registered reports and some may still make analytic suggestions upon seeing the data.
D. Tips of Trade
1. Researchers don’t have to use every open practice for every project. Pick what practices make sense for each project. Some practices may not fit some projects (e.g., if data collection timeline is fixed, registered report may not work).
2. Posting preprints makes your work available and citable much earlier. This can lead to earlier/more citations, invited talks, additional feedback, and easier access to your work from more readers. Tools for helping navigate preprints include sherpa romeo (searchable list of journal policies around access; https://v2.sherpa.ac.uk/romeo/).
3. Preregistration and registered reports represent a time shift in work and can ultimately save time. Registered reports essentially guarantee that data analysis will only have to happen one time (preregistration does not).
4. Many open practices were designed with quantitative research methods in mind. But this does not mean that they are irrelevant to qualitative research, although how they are implemented and what value they provide may differ in both magnitude and kind.

McBee, M. T., Makel, M. C., Peters, S. J., & Matthew, M. S. (2018). A call for open science in giftedness research. Gifted Child Quarterly, 62(4), 374–388. doi: 10.1177/0016986218784178
van der Zee, T., & Reich, J. (2018). Open education science. AERA Open, 4(3), 1–15. https://doi.org/10.1177/2332858418787466

Author

• Matthew C. Makel, University of Calgary