Confidence Interval for Standardized Effect Size in Cluster-Randomized Trials: Comparing Analytic and Bootstrap Methods

• In Event: Issues in Parameter Estimation in Complex Multilevel Models

Abstract

Although many methodologists have urged the use of effect size measures accompanying tests of statistical significance, discussions on obtaining confidence intervals (CIs) for multilevel effect sizes has been rare. In this paper we explore the bootstrap as a viable and accessible alternative for obtaining CIs for multilevel standardized mean differences (SMD). A simulation is carried out to compare 10 procedures for constructing CIs in terms of empirical coverage rates and length, and the results provide practical guidance for substantive researchers in constructing CIs for SMD in cluster randomized trials. Results showed that the bootstrap methods generally outperformed analytic methods when normality is violated and when the clusters are unbalanced, and the two residual bootstrap method performed best.

Author

• Hok Chio (Mark) Lai, University of Southern California