Response Ratio Effect Sizes for Single-Case Designs With Behavioral Outcome Measures

• In Event: 30.075 - The Meta-Analysis of Single-Subject Experimental Design Data: Methodological Challenges and Approaches

Abstract

Methods for meta-analyzing single-case designs (SCDs) are needed in order to inform evidence-based practice in special education, school psychology, and other fields. The most widely used outcome measures in single-case research are based on direct observation of behavior, which often take the form of rates or proportions. For studies that use such data, a simple and intuitive way to quantify the magnitude of treatment effects is in terms of proportionate change from baseline, using an effect size called the response ratio. Advantages of the response ratio effect size include that it is consistent with how many applied behavioral researchers conceptualize treatment impacts (Marquis et al., 2000) and that it is relatively insensitive to procedural measurement details (Pustejovsky, 2015). The response ratio is also closely related to other effect sizes that have occasionally been used in synthesis of single-case research, such as the Mean Baseline Reduction (Campbell, 2004) and Suppression Index (Marquis et al., 2000). These exceptions notwithstanding, the response ratio is seldom applied for meta-analyzing single-case research because supporting statistical methodology remains under-developed.
This paper describes statistical methods for estimating response ratios (along with corresponding variances) and for combining response ratio estimates using meta-analysis. The methods are based on a simple model for comparing two phases, where the level of the outcome is stable within each phase and the repeated outcome measurements are independent. Auto-correlation in the outcome measures will lead to inconsistent estimates of the sampling variance of the effect size. However, meta-analysis of response ratios remains possible by using robust variance estimation procedures (Hedges, Tipton, & Johnson, 2010; Tanner-Smith & Tipton, 2014) that are valid even when sampling variance estimates are inconsistent. The methods are demonstrated using data from a study of treatment for pica in children with developmental disabilities (Call, Simmons, Mevers, & Alvarez, 2015).
 

Author

• James Eric Pustejovsky, The University of Texas - Austin