Investigating Immediacy in Multiphasic Single-Case Experimental Designs Using a Bayesian Unknown Change-Points Model

• In Event: Bayesian Methods in Single-Case Experimental Designs

Abstract

Immediacy is one of the criteria for demonstrating strong evidence of causality in single case experimental designs (SCEDs, Kratochwill et al., 2013). A rapid change in the observations across phases indicates immediacy. However, no clear guidelines exist about decisively concluding the presence of immediacy. The only inferential method that investigates immediacy is the Bayesian unknown change-point model (BUCP). BUCP uses all data points to establish immediacy (Natesan & Hedges, in press). The shape of the posterior and the standard error of the change-point are used to investigate and quantify immediacy in bi-phasic AB designs.
However, AB designs for one subject cannot provide strong evidence of causality because they cannot be used to show three demonstrations of treatment effect unlike multi-phasic designs. Multi-phasic designs contain at least two change-points. Gibbs sampling algorithm will not compute efficiently for multi-phasic designs because the change-points in a multiple change-point model are simulated one at a time from full conditional distributions rather than from the joint distribution (Chib, 1998). Therefore, we extend the BUCP to multi-phasic SCEDs using Variational Bayesian (VB). We investigate the performance of the model using simulation and illustrate it for 13 datasets from 6 ABAB published studies.
VB is computationally more efficient than MCMC (Bishop, 2006). Each step is similar in complexity to Gibbs sampling but instead of drawing thousands of samples, VB typically sweeps through the model a few dozen times before convergence. We fit an intercept-only model to each phase where the boundary between the phases is assumed unknown. Consider the observed value at time y_t, which follows a normal distribution with the mean of y ̂_t - the expected value of the target behavior and standard deviation of σ_ε,
y_t~norm(y ̂_t,σ_ε^2 ).
Consider an ABAB design with 4 phases, that is, 3 phase changes at times p_1,p_2,and p_3, and a total of T time-points. The predicted value is,
y ̂_t=〖ρ*r_(t-1)+β〗_0p,
where ρ is the autocorrelation, r_(t-1)the residual at t-1, and β_0p the intercept of the linear regression model for phase p the time-point t belongs to. The residual is:
r_(t-1)={█(g_i,if 〖phc〗_t=1@y_(t-1)-y ̂_(t-1),if 〖phc〗_t=0 )┤

where 〖phc〗_t={█(1,if t-1≤p_i and t>p_i; i=1,2,3@0,otherwise )┤ and

g_i~norm(0,(σ_ε^2)/((1-ρ^2 ) ));i=1,2,3.

The intercepts β_0p:
β_0p= {■(β_01,if t≤p_1 @β_0i,if p_i
Priors:
σ_ε~gamma(1,1)

ρ~norm(0,100);bounded by (-1,1)

β_0i~ norm(0,100);i=1,…,4

All change-points were sampled from discrete uniform distributions ranging from time-point 1 to T and then ordered.
p_i~cat(pi);i=1,…,4

pi= (1/T,1/T,…,1/T);length(pi)=T

p_1 Three phase lengths (l = 5,8,and 10), four standardized mean differences (d = 1,2,3,5), and four autocorrelation values (ρ=0,0.2,0.5,0.8) were selected based on commonly occurring values in SCEDs (Maggin et al., 2011). Hundred datasets were generated for each combination. The intercepts of the baseline phases were set to 0 and the standard deviation within each phase σ=0.2.
The correct estimates of change-points had the maximum average probability (MAPr) for most conditions. Average probability weighted norm (APWN) decreased with ρ and l but increased with d. The trends are shown in Tables 1-2 and Figure 1. We will illustrate the results of real data analyses in the final paper (Table 3).

Authors

• Prathiba Natesan, University of North Texas

• Tom Minka, Microsoft Research

• Larry V. Hedges, Northwestern University