Case- and Time-Specific Effect Sizes (Poster 2)

• In Event: Analysis and Meta-Analysis of Single-Case Experimental Data: Methodological Developments

Abstract

Researchers using single-case experimental designs (SCEDs) may choose from a variety of effect size options including indices based on non-overlap, within-case standardized mean differences, between-case standardized mean differences, log response ratios, and percent of goal obtained. A limitation of these commonly used effect sizes is that they index effects at relatively coarse levels, such that there is one effect size per study, per case, or per AB phase pair. Thus, the researcher can only compute the effect for a single focal time or average it over time. In contrast, visual analysis of graphed raw data from SCEDs examines each point in the data stream, which enables researchers to study how the effect changes over time and to distinguish between effects that are constant, delayed or gradual, decaying, or transitory. We propose a way of estimating fine-grained (i.e., case- and time-specific) effect sizes, such that there is an effect estimate for each treatment phase observation.
To obtain these fine-grained effect sizes, a model fit to the baseline observations of a specific case is used to make predictions of what would have been observed had there been no intervention. Then, an effect estimate, delta_ij, is made at time i for case j by finding the difference between the observed treatment value, y_b_ij, and the projected baseline value, y_a_ij, at that same moment in time [i.e., delta_ij = estimate(y_a_ij) - y_b_ij for problem behaviors and delta_ij = y_b_ij - estimate(y_a_ij) for positive behaviors). Thus for positive behaviors, if the baseline is constant with values of 0, then delta_ij = y_b_ij. If the baseline is stationary, then delta_ij = y_b_ij - mean(y_b_ij), and if the baseline has a linear trend, then delta_ij = y_b_ij - (beta0_a_0j + beta1_a_1j).
We show how these fine-grained effect sizes can be standardized using existing approaches for case-specific effect sizes. This includes dividing delta_ij by a measure of the residual within-case standard deviation, finding the proportion of baseline observations that overlap with y_b_ij, making a log response ratio of y_b_ij to estimate(y_a_ij), or finding the proportion of the goal that delta_ij represents. To show the effect trajectory for each case, we provide procedures for graphing these fine-grained effect sizes. Graphing aids in the selection of an appropriate metric for the effect sizes, allowing comparisons of the information contained in the effect sizes to the information contained in the graphs of the raw data. We also show how to model these fine-grained effect sizes to obtain average effect estimates and to examine potential moderators, including those that vary with time (e.g., time in intervention, session-specific treatment fidelity) and those that characterize a case (e.g., severity of problem behavior at baseline).

Authors

• John M. Ferron, University of South Florida

• Lodi Lipien, University of South Florida

• Megan Kirby, University of South Florida