How to Preregister a Study

• In Event: Open Science Basics: Training for Education Researchers

Abstract

This presentation will overview why and how to pre-register a study for scholars seeking to implement that practice in their research studies. It will be designed for scholars who have limited experience with open science, but are looking to engage in more transparent research practices. Importantly, it will explore a number of “shades of gray” so that educational researchers can more readily see opportunities to partially benefit from certain aspects of pre-registration even for studies where they are not yet prepared to complete a full preregistration.

Preregistration is at its core designed to distinguish between confirmatory and exploratory results. Both are important to the progress of science, but when they are conflated, problems arise. Pre-registration specifies what you are setting out to confirm in your research versus what you discovered incidentally or along the way. The presentation will discuss the conceptual similarities and practical differences between pre-registration and registered reports and traditional approaches to educational research.

What is pre-registration?

Prereigistration is a research plan that is:
1) time stamped to , 2) cannot be changed, but can be updated with a new versions, 3) created before the study, and 4) submitted to a public registry.

Pre-registration will have the following components:
Study plan: Research Questions and Hypotheses, Data Collection Procedures, Manipulated and Measured Variables
Analysis Plan: Statistical Model, Inference Criteria, Power analyses, Potential interpretation of results
Secondary data analyses - variables listed from codebook, prior knowledge of data
Addendums - if needed, explain deviations from the original plan

Benefits of pre-registration
Protect against natural biases, selective reporting, and storytelling
Tool to communicate your work to others
Support robust research planning
Reminds your future self of your research goals
Facilitates a priori identification of potential problems with your research design while there is still time to address the issues
Increased transparency that can facilitate replication efforts

Pre-registered studies submitted as stage-1 registered reports have the potential to combat publication bias as the promise of publication comes prior to findings whether null or significant. They also increase the likelihood for reproducibility. Some journals offer Open Science badges to incentivize and recognize scholars.

Practical advice for using pre-registration:
Not every study warrants preregistration (papers reporting multiple studies may contain exploratory and preregistered studies)
Quasi-experimental, correlational, observational, secondary data, and qualitative studies can all be preregistered
Within a preregistered study some hypotheses might be prespecified; others might not be
You do not have to predict the future–if/then statements are okay (e.g., if randomization procedures produce disproportionate groups with respect to demographic variables, covariates for those variables will be included in the final regression model).

Authors

• Hunter Gehlbach, Johns Hopkins University
  Presenting Author

• Allison Lombardi, University of Connecticut
  Presenting Author