Bayesian Rate Ratio Effect Size for Count Data in Single-Case Experimental Designs

• In Event: 51.029 - Handling Methodological Complexities in Effect Size Calculation and Meta-Analysis of Single-Case Experimental Design Data

Abstract

Purpose: The purpose of the proposed study is to present a Bayesian rate ratio (BRR) effect size that can be used to measure treatment effect size of count data in small sample experimental designs such as single case experimental designs (SCEDs). To be considered as a contribution to SCED research practice, any new effect size estimation procedure should: (a) account for both autocorrelations and the scale of the data commonly used in SCEDs; (b) not require small sample corrections, and (c) produce reliable interval estimates of uncertainty. To my knowledge the effect size I propose here, the Bayesian rate ratio (BRR) effect size, is the first to meet these needs.
Theoretical framework: Bayesian methods can be deployed to overcome various analytical challenges presented by SCED data. A Bayesian approach does not depend on large sample or asymptotic theory (Ansari & Jedidi, 2000). Thus, small sample corrections will not be required for Bayesian effect sizes unlike the one proposed by Hedges, Pustejovsky, & Shadish (2012). Bayesian methods also allow more direct probabilistic interpretation of parameters than do classical methods and can address autocorrelation directly. Moreover, Bayesian models can handle model complexity such as count data and autocorrelation with considerable ease. Finally, Bayesian interval estimates of autocorrelated data are more accurate than frequentist estimates (Shadish, Rindskopf, Hedges, & Sullivan, 2013).
Methods: The Bayesian model used to analyze SCEDs in the present study is a Bayesian interrupted time-series (BITS) design where the intercepts and slopes vary by phase. I use count data and the dependent variable is modelled using Poisson regression. The rest of the time series follows a Poisson procedure with 1-lag autocorrelated errors (Harrop & Velicer, 1985). Bayesian Rate ratio is the ratio between the exponents of the means of treatment and baseline phases.
Data sources: Analysis of four datasets from peer-reviewed articles published in 2017 will be presented to demonstrate the model and BRR. Results from visual analysis and particularly percentage of non-overlap (NAP) index will be presented for comparison.
Significance: To date, the model I will present is the only inferential statistical model that estimates intercepts, slopes, autocorrelations, and BRR effect sizes that take into account autocorrelations for count data and that do not require small sample correction. The fact that the rate ratio estimation is made possible even for such short time-series is compelling evidence of the flexibility of Bayesian modeling.

Author

• Prathiba Natesan, University of North Texas