Effect Size, Heterogeneity, and Power in Meta-Analyses of Direct Replications: A Comprehensive Simulation Study (Poster 1)

• In Event: Sunday Poster Session 9:35 am
  In Poster Session: Applied Statistical Methods

Abstract

The field of psychology faces certain challenges such as overestimated effect sizes, false
positive findings, and replication failures, all stemming from inadequate power, publication
bias, and p-hacking. Over the past decade, the field of psychology has gone to great lengths
to scrutinize well-established findings via large-scale, preregistered multi-lab replication
projects. This study primarily investigates effect size variance (i.e., heterogeneity), an
often-overlooked contributor for replication failures. We emphasize the importance of
understanding and modelling heterogeneity using direct replications. The importance of
heterogeneity in power analysis for direct replications is also demonstrated. Overall, given
the inevitability of effect size heterogeneity, we highly recommend that researchers take
heterogeneity into consideration when planning for future multi-lab replication projects.

Author

• Naike Wang, Texas A&M University