Between-Case Standardized Mean Differences: Flexible Methods for Single-Case Designs

• In Event: Analysis and Meta-Analysis of Single-Case Experimental Data: Methodological Developments

Abstract

Single-case designs (SCDs) provide a flexible and rigorous technology for evaluating the effects of academic and behavioral interventions in educational and clinical settings, where individual cases serve as their own control (Horner et al, 2005; Gast & Ledford, 2018). Inferences about treatment effects for each case are typically drawn through comparing the data series of outcomes in the baseline phase with those in the treatment phase(s). Within the class of SCDs, the multiple baseline design is the most widely used (Barker et al., 2013; Shadish & Sullivan, 2011). Multiple baseline designs involve collecting data across several cases, which may represent different individuals or a single individual observed in different settings or for different behaviors. Their defining feature is that treatment is introduced at a different time point for each case. This staggered introduction ensures that treatment unlikely to be confounded by contemporaneous changes in the environment, thus protecting against history threats. Although visual analysis is the traditional primary method for analyzing SCD data (Kratochwill et al., 2010), there is a need for quantitative methods to synthesize results across a body of single-case research. Moreover, researchers might be interested to synthesize findings across studies that use different types of designs, which requires an effect size measure comparable across different designs (Pustejovsky et al., 2014). Hedges and colleagues (Hedges, Pustejovsky, & Shadish, 2012, 2013) developed such a measure, the between-case standardized mean difference (BC-SMD), which is theoretically comparable to the standardized mean difference from a between-groups experimental design performed on the same population and with the same outcome measure. Because of this comparability, BC-SMD effect sizes could be incorporated in a meta-analysis that includes SMD effect sizes from between-group designs, thus allowing the evaluation of intervention effects with a broader set of designs. Pustejovsky et al. (2014) proposed a general modeling framework for calculating BC-SMD effect sizes for multiple baseline designs across participants. This framework involves hierarchical linear models that allow for time trends and heterogeneous treatment effects across cases. Currently available methods for estimating BC-SMD are limited to across-participant multiple baselines, yet other, more complex designs may be encountered in practice. In this study, we extend the framework of Pustejovsky and colleagues (2014) and demonstrate how to estimate BC-SMD effect sizes from several variations of the multiple baseline design that are particularly relevant in school contexts. One such variation is the multiple baseline across behaviors or settings (e.g., Thiemann & Goldstein, 2004), where we show how to estimate BC-SMDs by pooling across replications of the design conducted with different participants. Another variation is the multi-level multiple baseline design, where each case consists of a group of participants, with outcomes measured on each participant (e.g., Bryant et al., 2018). A third variation is the multivariate multiple baseline design, where treatment effects are examined across participants for more than one outcome measure (e.g., Calder et al., 2020). For each of these variations, we describe methods for estimating a BC-SMD and illustrate our proposed approach by re-analyzing data from a published SCD study.

Authors

• Man Chen, University of Wisconsin - Madison

• James Eric Pustejovsky, University of Wisconsin - Madison

• David Klimgbeil