Analyzing Single-Case Experimental Count Data Using the Linear Mixed Effects Model: A Simulation Study

• In Event: 30.075 - The Meta-Analysis of Single-Subject Experimental Design Data: Methodological Challenges and Approaches

Abstract

Often SCED data are expressed as counts. To account for both the hierarchical and the count nature of the data, two frameworks can be combined: generalized linear modeling (GLM) and hierarchical or linear mixed modelling (LMM). Both frameworks have proven to be very flexible tools. From their most basic forms, they expand into more specialized models in a very clear and simple manner. By tweaking link functions, error distributions and moderator variables, one can create a generalized linear mixed model (GLMM) (McCullagh & Nelder, 1999) which is very well customized to the research questions and type of data.
One downside of the flexibility of the GLMM framework is that the framework is abstract and hence relatively complex to understand. Customizing a generalized linear mixed model requires a more general mathematical understanding of both the GLM and the LMM framework. Even though efficient estimation methods are available in many popular software packages and even though these models have proven their robustness and their power, they might be somewhat intimidating for social scientists to apply. Another difficult aspect of the GLMM framework is that the more sophisticated the model, the more information is needed to make sure the GLMM estimation converges. However, in SCED contexts typically a relatively small number of data points is available.
To address these issues, we study how well a simpler LMM for continuous data will perform. We set up a simulation study in which we simulate count data but analyze that data with a simple continuous linear model. The count data with a hierarchical structure are generated according to a two-level GLMM, assuming a Poisson distribution of scores within a phases.
We will analyze the simulated datasets by fitting the GLLM used for data generation, as well as by fitting a two-level LMM that assumes normality of the scores within phases.
The question is if and when this LMM is good enough for count data. Are there situations in which researchers do not have to bother with a complicated GLMM when dealing with hierarchical count data? To this purpose, we look at various simulation conditions. These conditions differ in the number of cases, the number of measurements within cases, the average baseline response, the average effect and the variance components. To analyze the goodness of fit of the LMM, we look at common goodness of fit criteria (e.g. AIC), the Type I error rate and the power.
In general, we expect that GLMM will outperform the LMM, but the LMM will still perform good if the expected count responses in baseline and/or treatment are relatively high, because relatively higher count responses should behave more continuously, i.e. for higher expected values, the Poisson distribution is well approximated by a normal distribution. The goal is to provide applied researchers with recommendations on the required design criteria (e.g. the required sample size or the required average count in the baseline and/or treatment phase) for analyzing count data with simpler LMMs.

Authors

• Lies Declercq, KU Leuven

• Wim Van den Noortgate, Katholieke Universiteit Leuven