Statistical Analysis and Effect Size Estimation for Single-Case Experimental Designs

• In Event: 61.033 - Single-Case Experimental Designs: Developments in Statistical Analysis, Effect Size Metrics, and Meta-Analysis Methods

Abstract

Single-case experimental designs (SCEDs), at the basic level, are short quasi-experimental time-series designs. In the simplest version, a subject is measured at several time points before the onset of an intervention to establish a baseline. Following the intervention, the subject is measured again at several time points. The patterns of observations before and after the intervention are compared to determine if the intervention is effective.

Single-case research methods provide a scientifically rigorous approach for documenting and advancing effective practices in education, behavior analysis, and psychology. Despite its long history and its applicability for drawing causal inference, single-case research has not advanced evidence-based practice in behavior analysis. Part of the reason can be traced to the fact that until very recently no acceptable statistical procedures were available for the analysis of and synthesis of results from single subject designs. In fact, attempts to apply statistical analysis to single-case research have not met with enthusiasm; descriptive visual analysis remains the predominant approach for assessing the effect of an intervention. However, well-established techniques such as spectral analysis that are appropriate with long time series and multivariate growth analysis methods are not appropriate in single subject designs where only a few observations are available. Linear model-based approaches that have been advanced have been criticized for failing to take into account the inherent dependency in the observations.

In this paper linear model based procedures are provided for the analysis of SCEDs. Linear as well as curvilinear regression lines are fitted to the pre- and post-intervention observations and compared. The dependencies in the observations are modeled using autoregressive schemes. An effect size measure is defined in terms of the average difference between the pre- and the post-intervention lines. The advantage of this effect size measure is that it allows for differing intercepts and slopes of the two regression lines. In the case where several subjects are available, multilevel models are fitted to obtain a d-type effect size measure where the between and within group variances are used for standardizing the effect size measure. This multilevel procedure is employed to quantify the overall impact of the intervention for the entire study. The applicability of the procedure for meta-analysis of SCEDs is described.

The distribution of the unstandardized as well the standardized effect size is derived using Bayesian as well as classical statistical methods. A Bayesian Markov chain Monte Carlo (MCMC) method is developed to obtain the posterior distribution of the effect size measure. It is shown that that the Bayesian methods used also provide improved estimation of the autoregressive parameters. Finally, the procedures are extended to handle nonlinear data where the data are discrete. Logistic and Poisson models are applied for the analysis of non-continuous data. Effect size measure similar to that defined for the linear data are provided for nonlinear data. Bayesian procedures for the analysis of such data are provided and illustrated with data obtained in SCED.

Authors

• Hariharan Swaminathan, University of Connecticut

• Jane Rogers, University of Connecticut