Combining Several Types of Single-Case Experimental Designs Using a Three-Level Meta-Analysis

• In Event: Handling Methodological Issues in the Analysis of Single-Subject Experimental Design Data

Abstract

The purpose of the study is to further extend the multilevel meta-analysis of single-subject experimental designs (SSEDs) by including complex SSEDs such as alternating treatment designs (ATDs) and ABAB reversal designs in addition to the multiple-baseline across participants design (MBDs). While in recent years, there is attention for the synthesis of MBDs using the two-level model (Ferron, Bell, Hess, Rendina-Giobioff, & Hibard, 2009; Van den Noortgate & Onghena, 2003) or the three-level model (Owens & Ferron, 2010; Moeyaert, Ugille, Ferron, Beretvas, & Van den Noortgate, in press), combining data from ABAB designs and ATDs hardly received any attention. Combining data from different designs studying the same research question increases both the validity and reliability.
The method we propose to combine data from these designs involves three steps. First we suggest calculating case-specific effect sizes using ordinary least square regression. Note that in the ABAB design, an immediate treatment effect and a treatment effect on the slope for the first and the second AB pair is calculated (Kazdin, 2011). Also for the ATDs an immediate treatment effect and a treatment effect on the slope for the first and the second treatment is calculated (Kazdin, 2011). Whereas we have one immediate treatment effect and one treatment effect on the time trend per MBD, we have two immediate treatment effects and two treatment effects on the slope for the ABAB designs and the ATDs. In a second step we standardize the case-specific effect sizes by using the estimated within-case residual variance (i.e., obtained by the OLS analysis). In a last step, we suggest conducting two separate three-level meta-analyses or one multivariate three-level meta-analysis to estimate the average immediate treatment effect and the treatment effect on the slope, and their variation across cases and studies. A predictor indicating the design type is included at the third level of the three-level model because (part of) the variation in outcome scores between studies might be explained by the design.
We illustrated the proposed method using a published meta-analysis of single-cases including the three types of SSEDs. We also performed a simulation study to empirically validate the suggested method. We simulated data using the three-level model proposed by Van den Noortgate and Onghena (2003), under a variety of realistic conditions, varying the sample sizes at each of the levels (i.e., the number of measurement occasions per case, the number of cases per study, and the number of studies), the between-study, the between-case variance, and the within-case variance, the size of the immediate treatment effects, and the treatment effects on the slope. We simulated 2,000 datasets for each condition. By looking at the bias, the mean squared error, the standard error, the confidence intervals of the 95% significant level and the power, we evaluate the three-level meta-analysis of several types of SSEDs. In addition to the analysis across designs, we also perform the analysis per design type which might be of interest when the overall analysis gives biased results.

Authors

• Mariola Moeyaert, University at Albany

• Maaike Ugille, K.U. Leuven

• Patrick Onghena, KU Leuven

• Wim Van den Noortgate, Katholieke Universiteit Leuven