Effect Sizes for Single-Case Research and Meta-Analysis Involving Single-Case Studies

• In Event: Methodological Dilemmas Encountered in Analyses and Meta-Analyses of Single-Case Design Research

Abstract

Busk and Serlin (1992) defined three approaches for effect-size measures for single-case studies. Some researchers attempting to use their procedures have encountered difficulties due either to miscalculation of pooled standard deviations (SD) or due to zero variances at either baseline, intervention, or both.
The miscalculations may be a misinterpretation of how to compute the pooled SD. Koutsoftas, Harmon, and Gray (2009) were interested in obtaining effect sizes for a single-case multiple baseline study that was conducted with 34 low-income preschool students to increase their phonemic-awareness skills. There were two or four baseline observations, six intervention phase observations, and two or four post-intervention observations. The effect size was computed using one of Busk and Serlin’s (1992) approaches resulting in a total of six observations. The SD was computed erroneously across all 12 observations of each individual instead of pooling the SDs. The researchers; error may have arisen from the lack of variability at baseline, post-intervention, or both. The authors acknowledged that individual effect sizes could not be calculated because of zero variance, as the observations were identical for all three phases. In this case, the effect size is zero because observations have not changed.
When either the baseline or intervention variance is zero, the appropriate SD is the square root of MSW where the degrees of freedom for the MSW is based on the within-subject degrees of freedom, (the number of observations minus two) because one degree of freedom is lost for each phase mean (baseline and intervention). We will illustrate the computation of effect sizes for the Koutsoftas et al. (2009) study.
Meta-analysts calculate an effect size for each study and each distinct outcome measure. Busk and Roberts’ (2009) recommended obtaining overall study effect sizes by pooling means for baseline and for intervention and by pooling the standard deviations to obtain the effect-size measure. If there are zero variances, then the meta-analyst would obtain the overall effect size using the square root of the MSW computed across all individuals. Here, the degrees of freedom for the MSW would be obtained by adding the within-subject degrees of freedom. For example, in the Koutsoftas et al. (2009) study, there were 34 preschool children observed six times across baseline and postintervention phases, resulting in six minus two (i.e., 4) degrees of freedom within subject and the degrees of freedom for the MSW of 34*4 or 136. The overall effect size for the Koutsoftas study is 2.34.
Because the zero variance situation may occur when obtaining the study effect size for all of the studies used in the meta-analysis, the meta-analyst should address the discrepancy that would arise unless conducting a hierarchical linear model (HLM) analysis, then the effect size for each study would be computed using the square root of the total MSW for the denominator and the difference in pooled means for baseline and for intervention phases for the numerator. An example of such a meta-analysis situation will be illustrated in the presentation.

Authors

• Patricia L. Busk, University of San Francisco

• Ronald C. Serlin, University of Wisconsin - Madison