Go Your Own Way: A Critical Review of Qualitative Open Science Practices Outside of Education

• In Event: A Critical Perspective of Conducting Open Science Qualitative Education Research

Abstract

With the recent calls for increased transparency and rigor come calls for the incorporation of open science practices. The majority of these open science practices have been used within quantitative research but do not align practically or with the values of qualitative researchers. For example, popular open science practices include practices to prevent selectively finding and publishing statistically significant results (e.g., Head et al., 2015) that do not apply to qualitative research projects. While much debate in educational research exists over the distinction between qualitative and quantitative research, with many researchers stating the polarization of the two paradigms does more harm than good (NRC, 2002; Salomon, 1991), both qualitative and quantitative educational researchers face common issues of validation, generalizability, and standards of quality, yet researchers from the two paradigms should resolve them in different ways given the different assumptions made in each of the two camps (Salomon, 1991). Additionally, these open science practices that have been used with qualitative research emerged outside of education and may not apply in the same way as they do in other fields. Understanding differences in education research from that of other fields while still maintaining high standards becomes complex yet important for moving the field forward. As such, the purpose of this paper is to review qualitative open science practices from other fields to inform the conversation about what practices could and/or should not be used by educational researchers.
We conducted a search to identify scholarly sources related to conducting open science qualitative research. To provide a broad list of results, scholarly sources included journal publications, pre-prints, online discussions, resources (i.e., checklists, podcasts, workshops), and websites that cited academic publications and concepts that would be relevant to an educational context. We searched open science websites (e.g., Open Science Framework, Open Educational Resources) using terms related to qualitative research. We also searched Google and Google Scholar for gray literature, such as blogs and op-eds. Following the initial search, we utilized snowball search techniques to examine the references of the initial sources and contact authors to identify publications we had not yet found. Finally, we sent the list to researchers within the realms of open science, education, and qualitative research to identify other potential resources.
Results included items related to general reporting standards (seven), researcher positionality (two), preregistration (six), registered reports (two), open materials (two), open analysis (five), open data (13), data repositories (11), preprints (one), and ethical considerations (six). These resources came from multiple fields, with political science, psychology, and qualitative methods having the most results within our list. Using these search results as a foundation, we conclude our paper with possibilities for qualitative educational researchers to consider for their practices. We also argue about the practical difficulties and necessity of each practice with a particular focus on equity.

Author

• Rachel Renbarger, Duke University