Multilevel Modeling in Single Case Studies with Count and Proportion Data

• In Event: Analysis and Meta-Analysis of Single-Case Experimental Data: Methodological Innovations and Complexities

Abstract

Single case experiment design (SCED) plays an important role in research on behavioral modification and evaluation of a given intervention. As a flexible statistical approach to analyze SCED data, linear mixed-effects modeling (LMM) usually assumes that the residuals should be normally distributed. However, frequency count of target behaviors and proportions of intervals where target behaviors are observed are common types of outcome measurements in SCEDs. In such cases, the normality assumption in LMM is often violated (Declercq et al., 2019; Shadish, Kyse, & Rindskopf, 2013; Shadish & Sullivan, 2011).
Generalized linear mixed modeling (GLMM) is very flexible to accommodate various types of non-normal data. However, this approach may be difficult for empirical researchers to apply as it requires a general mathematic understanding. In the present study, we will provide a colloquial illustration of important concepts in data consideration, model specification and results interpretation for GLMM in comparison with the counterparts in LMM.
The main differences between GLMM and LMM lie in the following two areas. First, the specification of the distribution for the response variable varies. For frequency count, Poisson distribution is appropriate while for proportions binomial distribution is adopted. However, unlike the normal distribution in LMM, the mean and the variance for Poisson and binomial distribution are not independent. When the data has more variation than would be predicted by Poisson or binomial models, this phenomenon is called overdispersion which leads to biased estimates and standard errors if it is not accounted for (Shadish et al., 2013). In a recent review, Pustejovsky, Swan, and English (2019) provided empirical evidence of different degrees of overdispersion in SCED data. Therefore, alternative models that can account for overdispersion such as negative binomial model for frequency count, Beta binomial model for proportions and observation-level random effects model (OLRE) for both types of data should be considered (Hilbe, 2011). Second, nonlinear link functions are used for frequency count (log function) and proportions (logit function) in GLMM while the identity link function is adopted in LMM. This plays a key role in the differences of interpretation between GLMM and LMM. That is, unlike the LMM, the conditional effects (case-specific effects) are not the same as the marginal effects (population effects) in GLMM. Finally, the scales of treatment effects in LMM (additive effects) and GLMM (multiplicative effects) are different. Transformation of effect sizes on different scale is probably of interest for meta-analysis. However, the justification of transformation using either conditional effects or marginal effects should be first considered.
For demonstration purposes, we will provide two examples of SCED data. The first example involves overdispersed frequency count data. We will fit an LMM, Poisson model, negative binomial model and OLRE model to the data. The second example involves overdispersed proportions data analyzed using LMM, binomial model, Beta-binomial model and OLRE model. To evaluate performance of these models, a small Monte-Carlo simulation will be conducted with conditions based on the empirical setting.

Authors

• Haoran Li, Texas A&M University - College Station

• Wen Luo, Texas A&M University - College Station

• Kwok Hap Lam, Texas A&M University - College Station

• Eunkyeng Baek, Texas A&M University - College Station

• Christopher G. Thompson, Texas A&M University - College Station