Characterizing an Effect Size by Its Percentile in the Meta-Analysis Estimated Population Distribution of Effects

• In Event: Methodological Innovations for Conducting Research Syntheses

Abstract

Effect size characterizations are best made in reference to effect sizes found in the literature. We propose using the results of a random-effects meta-analysis to inform the characterization of an effect size. With random-effects meta-analysis, a population distribution of effects is estimated. The estimates of the mean and variance are uncertain, however, so we propose using the prediction distribution (used to estimate the prediction interval) to serve as the reference distribution when characterizing an effect size. Doing so, the percentile of an observed effect and the limits of the effect size’s 95% confidence interval within the prediction distribution are calculated. We demonstrate this procedure with an example and provide recommendations (and cautions) for meta-analysts and researchers conducting a single study.

Authors

• Peter Boedeker, Baylor College of Medicine

• Gena Nelson, University of Oregon

• Hannah Carter, Boise State University