Implications of Using Multilevel Latent Class Analyses on School Policy Interventions

• In Event: Problems With Interpretations of Multilevel Data: Extending Research Beyond Hierarchical Linear Modeling

Abstract

Hierarchical Linear Modeling (HLM; Raudenbush & Bryk, 2002) has been the primary approach to modeling nested data. Other (e.g., bootstrap methods that resample clusters of observations, clustered robust standard errors, or Multilevel Latent Class Analysis; MLCA) models often have been ignored despite their potential usefulness in deriving more meaningful and accurate results from nested data. In this study, we demonstrate an application of MLCA to examining survey data administered to 31, 966 12th grade students from across 332 schools in a large Midwestern state of the United States.
MLCA, a less frequently used method than HLM, accounts for the nested structure of data by allowing latent class intercepts to vary across Level 2 units and examines how these units affect latent classes or groups (e.g., students) at Level 1. These random intercepts allow the probability of membership in Level 1 to vary across Level 2 units (Henry & Muthen, 2010). In MLCA, the dependent variable is latent rather than observed, providing us with an advantage of combining this measurement model with analyses such as structural equation modeling and growth modeling. Additionally, observed dependent variables can be continuous, censored, binary and ordinal, allowing for a more flexible approach to multilevel analyses (Muthén & Muthén, 2012).
We used MLCA to estimate seven constructs measuring individual, family, and school related beliefs and attitudes (e.g., mental distress, family support, school safety and climate and teacher and community support), as well as individual (e.g., age, grade point average, and gender) and school predictors (e.g., school’s socioeconomic tier, urban/ rural locations, and pupil attendance). We compared results from MLCA to individual-level Latent Class Analysis models, which ignore data nesting. Our findings indicated that random multilevel models are statistically more accurate than single level analyses when there is data nesting. Our results have implications on the types of policy interventions that can be implemented based upon results from survey questions. To illustrate, the question: “Are hallways safe?” may yield different results depending on whether single- versus multilevel analyses methods are used. Results from single level analyses may indicate that hallways are safe on average whereas results from multilevel analyses – which considers the nature and type of schools from which the data were collected - may indicate lack of hallway safety in some (e.g., rural) schools. Results derived from single level analyses may not prompt any interventions given that hallways seem (on average) safe whereas results from multilevel analyses may foster interventions (e.g., instituting hallway monitors for rural schools). Such degree of specificity and contextualization can be useful when designing policy interventions to generate interventions that are more targeted to the groups or schools in need. Therein lays this study’s key contribution, which is to guide policy and intervention by obtaining more accurate and meaningful results from data analyses. This study is unique in that most studies fail to make these types of contextualized analyses thus leading to broad and general applications and policy interventions, which may be of limited use and generalizability for particular subgroups.

Authors

• Julio C. Cabrera, University of Minnesota

• Stacy R. Karl, University of Minnesota - Twin Cities

• Michael C. Rodriguez, University of Minnesota

• Maria Elena Oliveri, Educational Testing Service