Multilevel Meta-Analysis of Single-Case Experimental Designs Using Robust Variance Estimation

• In Event: Innovations in Experimental Design and Causal Inference

Abstract

One approach to synthesizing single-case experimental designs (SCEDs) uses multi-level meta-analysis (MLMA). However, MLMA relies on having accurate sampling variances of effect size estimates for each case, which may be infeasible due to auto-correlation in the raw data series. One possible solution is to combine MLMA with robust variance estimation (RVE). Another possibility is to use ordinary least squares (OLS) methods with RVE. This simulation study evaluates the performance of these synthesis methods for SCEDs consisting of auto-correlated count outcomes, for several different effect size metrics. Results demonstrate that MLMA with RVE performs properly for some effect size metrics but not for others, providing important guidance for practice as well as direction for further methodological development.

Authors

• Man Chen, University of Texas at Austin

• James E. Pustejovsky, University of Wisconsin - Madison