(Re)Thinking About Transparency and Rigor in Qualitative Open Science

• In Event: A Critical Perspective of Conducting Open Science Qualitative Education Research

Abstract

Discussions around rigor and transparency within the open science movement have largely come from a positivist/quantitative perspective that focuses on transparency of outputs, namely open data, open materials, open code, and open access (to manuscripts) (Hagger, 2019; Lyon, 2016; Powers & Hampton, 2018). Fecher and Friesike (2013) describe five schools of open science, but notably do not discuss transparency (which is arguably about the research process) in depth; this is different from accessibility (which is arguably about research products). The desire of many open science proponents to assign strict guidelines (e.g., always share data) may create problems for those working with qualitative data and those with prolonged engagement with their participant community. Defaults can be useful because they automate processes and reduce our cognitive load when making decisions, but they can also be dangerous for these same reasons, and can result in unchecked assumptions and sloppy work (Sakaluk, 2021).
Previous work within qualitative research methods suggests it may be helpful to think of the relationship between transparency and rigor (e.g., Billups, 2014; Davies & Dodd, 2002; Finlay, 2006; Mill & Ogilvie, 2003; Rolfe, 2006), and how transparency in all aspects of our research (where applicable) can lend itself toward projecting and confirming the rigor of our work. Qualitative researchers have long explored the potential for researchers to increase transparency through laying bare the research process using strategies like thick description (Henry, 2015; Morse, 2015) and reflexive practices like positionality statements (Guillemin & Clancy, 2013; Gillam, 2004; Rooney, 2013; Savin-Baden & Major, 2013). Practices like these can allow readers to make decisions around the transferability of findings, check for misinformation and anticipate potential points of confusion, and understand where the researcher may have influenced the research process and whether this may impact the validity or strength of their interpretations. Opening the “black box” of the research process in this way also allows readers from all educational backgrounds to better understand how research is done, what kinds of decisions are made during the course of a research project, and learn best practices within the researchers’ respective field(s).
This presentation will review discussions and guidelines for transparency within the open science movement and qualitative research spaces and explore places of agreement and tension, with an eye for what the open science movement may be able to learn from established best practices in qualitative methods. Participants are encouraged to consider what open science guidelines or suggestions may be helpful in their own work, what guidelines or suggestions are not compatible with their work, and what practices in their own work may not be currently considered within open science practices, but could be helpful in achieving the goals of the open science community around transparency, rigor, and accessibility.

Author

• Crystal N. Steltenpohl, The Dartmouth Institute for Health Policy & Clinical Practice