Standardized Mean Differences: Not So Standard After All

• In Event: Advancing Meta-Analytic Methods

Abstract

Meta-analyses often use standardized mean differences (SMD), (e.g., Cohen’s d and Hedges’s g), to compare treatment effects. However, these SMD indices are highly sensitive to the within-study sample variability used for their standardization, potentially distorting individual effect size estimates and compromising meta-analytic conclusions. This study introduces harmonized standardized mean differences (HSMD), a novel sensitivity analysis designed to evaluate and address such distortions. The HSMD harmonizes relative within-study variability across studies by employing the coefficient of variation (CV) to establish empirical benchmarks (CV quartiles). SMDs are then recalculated under these consistent variability assumptions. Applying this framework to meta-analytic data reveals the extent to which (original) effect sizes and pooled results are influenced by initial, study-specific standard deviations to standardize mean differences.

Authors

• Juyoung Jung, University of Iowa

• Ariel M. Aloe, University of Iowa