BackHome
Refresh: OffView Personal Schedule
Paper Summary
Share...

Direct link:

A Systematic Review of Meta-Analyzing Single-Case Data Using Multilevel Modeling

Sun, April 7, 3:40 to 5:10pm, Metro Toronto Convention Centre, Floor: 800 Level, Room 801A

Abstract

In the past decade there has emerged a growing interest in meta-analysis of single-case experimental design (SCED) (Shadish, 2014; Shadish, Hedges, & Pustejovsky, 2014). SCEDs have been frequently employed in various disciplines to assess the effects of interventions and treatments. SCEDs provide researchers with a feasible alternative to group designs with large sample sizes, while also allowing researchers to study treatment effects on special populations such as children with autism or developmental disability (Shadish, 2014; Shadish & Rindskopf, 2007; Smith, 2012). However, SCED has a major issue of generalizability due to its its use of a small number of cases. Meta-analysis can be used to enhance the generalizability of SCEDs. Using meta-analysis allows researchers to quantitatively synthesize results of past research, thus, providing evidence to support best practices (Beretvas & Chung, 2008; Petit-Bois et al., 2016; Tincani & De Mers, 2016). SCED meta-analysis aggregates the treatment effect across studies which enhance generalizability, while also allowing researchers to examine how the treatment effect relating to specific individuals within a study.
Although there are several statistical methods available to synthesis SCED data, there is no consensus on the best methods to synthesize these data (Van den Noortgate & Onghena, 2008). The approach of multilevel modeling has recently been suggested for combining SCED data, with promising results (Owens & Ferron, 2012; Ugille, Moeyaert, Beretvas, Ferron, & Van den Noortgate, 2012; Petit-Bois et al., 2016). For example, the fixed effect estimated in a three-level model tends to be reliable and reasonably unbiased when using small sample size.
The current study responds to the need to provide guidance or information to both applied single-case researchers and methodologists regarding an overview of meta-analysis of single-case data using multilevel modeling. The multilevel modeling approach provides a large degree of flexibility in modeling the data (Van den Noortgate & Onghena, 2003). As a consequence, this added flexibility requires researchers to make a series of methodological decisions when specifying the model to approximate the data. Such decisions are critical as they can affect an accuracy of parameter estimates if a statistical model is not correctly specified. Although there is an increasing number of studies to examine misspecifying issues of meta-analysis of single-case data in multilevel framework, a general overview or guidance is missing. Such a guidance will provide not only critical information regarding best practice of specifying model for applied researchers, but will also inform areas of interest for methodologists to reduce a gap between applied researchers and methodologists. This paper systematically reviews various methodological characteristics of multilevel modeling of SCED meta-analyses published between 2000 and 2018, including, but not limited to, number of levels, designs, standardization, random model, autocorrelation, analyses of raw data or effect sizes. A systematic review of published studies provides guidance to applied researchers on how to specify models that are most appropriate in various situations, as well as informs methodologists how applied researchers specify models for their data, and how often findings from simulation studies have been utilized.

Authors