Bayes Factor for Analyzing Single-Case Experimental Designs: Comparing Several Models Related to ABAB Design Data (Poster 2)

• In Event: Methodological Innovations and Complexities in the Analysis and Meta-Analysis of Single-Subject Experimental Design Data

Abstract

Single-Case Experimental Design (SCED) has been utilized in various research fields such as clinical psychology, special education, and so on. In recent years, interests about research using SCED have been growing. ABAB design is an extension of AB design, also called “withdrawal design” or “reversal design”. If we find similar changes in the dependent variable between two AB pairs, we can be confident in the intervention effects.
Bayes factor is an index used for Bayesian hypothesis testing (e.g., Wagenmakers, Lodewyckx, Kuriyal, & Grasman, 2010) and is used for relative comparison of two hypotheses, corresponding to the null and alternative hypotheses in traditional statistical hypothesis testing. de Vries & Morey (2013) developed the models that could calculate Bayes factors from AB design data. Their models can deal with serial dependency of SCED data by assuming AR(1) model for the error term. The model proposed by de Vries & Morey (2013) can only be applied to data from AB designs. Therefore, we extend the model so that it can be applied to data from ABAB design. We set up two models: Model A treats the intervention effects by grouping two intervention phases together and Model B includes three parameters of change in level. We use Bayes factors not only for Bayesian hypothesis testing of intervention effects, but also for comparing two proposed models.
In this study, the following two models were proposed for the ABAB design, referring to Moeyaert et al. (2014).
Model_A:y_i=μ_0+σ_Z*δ*x_i+ε_i
Model_B:y_i=μ_0+σ_Z*δ_1*A1B1_i+σ_Z*δ_2*B1A2_i+σ_Z*δ_3*A2B2_i+ε_i
Setting of the prior distribution for Model A was based on de Vries & Morey (2013), for Model B was based on Rouder et al. (2012). The data used to calculate Bayes factor was the data from Uchida and Tanji (2021). Bayes factor was calculated on this data using R packages (rstan, polspline, ks, bridgesampling).
For Model_A, mean(δ)=1.48 and BF_Model_A=12.989. As for Model_B, mean(δ_1)=0.63, mean(δ_2)=-1.13, mean(δ_3)=1.36 and BF_Model_B_for_δ_3=2.227, BF_Model_B_for_δ_1&δ_3=2.368. On the other hand, BF_Model_B_for_δ_1=0.786. By Bridge sampling method, BF_Model_A_vs_Model_B=36.852.
In this study, we extended de Vries & Morey (2013) applicable to ABAB design and applied to the real dataset. The change in level of the dependent variable from A1 to B1 was not so clear, which was also reflected in the mean of the posterior distribution of δ_1. On the other hand, the change in level of the dependent variable from A2 to B2 was quite clear, which was also reflected in the mean of the posterior distribution for δ_3. Bayes factor obtained by Bridge sampling method supported Model_A for data from Uchida & Tanji (2021).
Further development of this study could be the application to various SCED other than AB and ABAB designs through flexible Bayesian statistical modeling. On the other hand, there are still issues that need to be carefully considered, such as the assumptions of prior distributions of parameters.

Authors

• Tsuyoshi Yamada, Yokohama City University

• Kensuke Okada, University of Tokyo