Applicability of an Effect Size for Single-Case Studies That Indexes Progress Toward a Goal

• In Event: Analysis and Meta-Analysis of Single-Case Experimental Data: Methodological Innovations and Complexities

Abstract

The most commonly used type of effect size measure in SCEDs are non-overlap indices, such as the percent of nonoverlapping data (PND), nonoverlap of all pairs (NAP), and Tau-U (Parker, Vannest, Davis, & Sauber, 2011). These indices also are known to have limitations, such as the inability to differentiate between effects of different sizes when there is no overlap. Another type of SCED effect size is the standardized mean difference (SMD; Busk & Serlin, 1992), and its trend-adjusted regression and multilevel modeling counterparts (Pustejovsky, Hedges, & Shadish, 2014; Swaminathan, Rogers, & Horner, 2014). These too have limitations. When the baseline standard deviation is used to standardize (i.e., within-case standardization, SMDW) the effect size cannot be computed for cases with no baseline variation; and when between-case variation is used for standardization (SMDB), the effect size cannot be computed for studies with only one or two cases. Finally, the response ratio, obtained by dividing the treatment phase mean by the baseline phase mean, or the corresponding log-response ratio, can be used as a measure of effect, but a response ratio cannot be computed if the mean baseline value is zero, and is limited to ratio-level measures (Pustejovsky, 2018).

An alternative way of measuring effect is to gauge progress toward a goal. Let γ be the desired level of the behavior or goal. Let α be the expected level of behavior in the absence of intervention, and let β be the level of the behavior obtained during intervention. The percent of goal obtained (PoGO) is defined for contexts where the goal is to increase behavior as PoGO = ((β – α)/(γ – α))*100. In contexts where the goal is to decrease the behavior, PoGO = ((α – β)/( α – γ))*100.

In the special case where the goal is the extinction of a problem behavior (γ = 0), and α is estimated as the baseline mean, and β is estimated as the treatment phase mean, PoGO is a function of the response ratio. Specifically, for this special case PoGO = ((α – β)/( α – 0))*100 = (1 – (β/α))*100, where β/α is the response ratio. However, in other contexts where the goal is not 0, there are meaningful differences between PoGO and the response ratio.

In this study we assessed the applicability of PoGO by using it to re-analyze data from previously published single-case studies. We compared the results of PoGO to visual displays of the behavior and to other effect sizes (Tau-U, NAP, the SMD, and the log-response ratio). The re-analyses found PoGO aligned well with the effects seen in graphical displays, was easy to interpret, and could be used when baseline values were zero or the study included a single case. Limitations of PoGO and the need for additional research are discussed.

Authors

• John M. Ferron, University of South Florida

• Howard Goldstein, University of South Florida

• Arnold Olszewski, Miami University

• Lodi Lipien, University of South Florida