Meta-Analysis of Monte Carlo Simulations Examining Class Enumeration Accuracy With Mixture Models

• In Event: The Utility of Mixture Modeling to Understanding Heterogeneous Effects in Educational Studies

Abstract

The increased use of mixture models has resulted in many methodological papers consisting of Monte Carlo simulations examining the performance of the statistical technique under a variety of scenarios. One major focus of methodological simulation studies with mixture models is the performance of different indices used to determine the number of classes to extract, which is referred to as class enumeration. While some findings are clear with respect to class enumeration performance, not all indices perform similarly across manipulated conditions and/or models examined in the simulation studies. As such, we conducted a meta-analysis of simulation studies examining the performance of class enumeration indices with mixture models.

Meta-analysis techniques are used to quantitatively synthesize the findings from a sample of empirical research studies assessing similar hypotheses. The quantitative syntheses generally involve summarizing average treatment effects or relationships among variables as well as the variability of the treatment effects or relationships of variables across the sample of studies. Synthesizing the results from Monte Carlo simulation studies is a unique way to understand the performance of methodological techniques under various manipulated conditions.

Method and Data Sources
We searched databases, including Education Resources Information Center (ERIC), PsycINFO, PsycARTICLES, Psychology Behavioral Sciences Collection, Wiley online library, SAGE journals, ProQuest Dissertations & Theses Global, and Taylor and Francis Online using the following key words: ("class enumeration" OR "class number" OR "enumeration index" OR enumeration) AND ("growth mixture model*"OR "latent mixture model*" OR "mixture model*" OR "latent class" OR "finite mixture model*") AND ("monte carlo" OR simulation). Searches were limited to studies published in English from 1990 to May 2020. This search resulted in 629 records identified for screening.

After removing 26 duplicates, 603 abstracts were first screened for inclusion. We included studies if the study: 1) included a simulation; 2) used latent variable modeling; 3) provided quantitative data; and 4) reported class enumeration outcomes. This resulted in 118 full-text articles reviewed for eligibility. Of the 118 articles, 67 did not match our criteria, 18 were applications of mixture models (not simulations), and three studies did not include tabled results. In the end, 30 articles were included in the meta-analytic review. The 30 articles were each coded by two reviewers for the following information: type of mixture model; type of enumeration indices examined; the number of enumeration indices examined; true number latent classes in the population; sample size; prevalence; class separation; and enumeration accuracy rates.

Results and Scholarly Significance
The analysis will include the following variables as moderators to test for their impact on class enumeration accuracy: type of mixture model; type of enumeration indices examined; the true number latent classes in the population; sample size; prevalence; and class separation. It is hoped that the findings will help inform applied researchers about the performance of different indices as they interact with other variables (e.g., type of mixture model; number of classes) in terms of their accuracy. Implications of using meta-analytic techniques with simulation studies, which are the ultimate experimental studies, will be discussed.

Authors

• Tiffany A. Whittaker, University of Texas at Austin

• Jihyun Lee, University of North Texas

• Devin Dedrick

• Christina Munoz