Executing a Registered Report for Systematic Review and Meta-Analysis: An Illustration

• In Event: Methodological Approaches for Equity-Oriented Evidence Synthesis

Abstract

Open science is equitable science and the necessary pathway to advancing rigorous and replicable best practices in research (Nosek et al., 2015; Pigott & Polanin, 2020). One relatively new form of open science practice is a Registered Report, wherein the project conceptualization, methods, and proposed analyses are pre-registered and subject to peer or board review prior to data collection and analysis (Page et al., 2021). A Registered Report format is designed to optimize equitable data treatment and analyses. While increasingly common in primary studies (Nosek et al., 2014), Registered Reports are virtually non-existent in the evidence synthesis literature (Chambers & Tzavella, 2022). There are many benefits to registering a meta-analysis, including increased visibility of study intentions, reduction of redundant effort in a given field of inquiry, and promoting access to unpublished studies and data as well as amassing staffing and project support (Chambers & Tzavella, 2022; Open Science Collaboration, 2015). There are also many intricate challenges to registering a meta-analysis, including identifying the codes in advance of searching the data through extraction, articulating complete research questions and hypotheses with empirical derived estimates of projected power needs, and accounting for the unknown in data extraction.

The present paper presents an applied illustration with a recently executed, first of its kind, Registered Report of a meta-analysis published by Child Development (Cipriano et al., 2023). This systematic review and meta-analysis of the evidence for universal school-based social and emotional learning (SEL) interventions serving students in Kindergarten through twelfth grade followed the contemporary Preferred Reporting Items for Systematic Review and Meta- Analysis PRISMA guidelines (Page et al., 2021) and adhered to a predetermined, peer-reviewed protocol to strengthen transparency through pre-registration (Nosek et al., 2015; Pigott & Polanin, 2020). The Registered Report took three years from inception to publication, and leveraged Open Science Framework (OSF) as a resource and public communication hub for project decision making and progress. Upon project completion, we also shared all necessary analytic files and the data, instructions, and code for analysis on OSF to support transparency and replication. Presentation will systematically describe how to effectively address challenges to registration in meta-analyses to support increasing open science practices in evidence syntheses at large. Further, the presentation will highlight the benefit of open science data treatment in the service of hot button topics in education like SEL in support of advancing equity through evolved educational research methods.

Author

• Christina Cipriano, Yale University