The Impact of Ignoring Cross-Classified Data Structures When Estimating Cross-Classified Logistic Models

• In Event: Cross-Classified and Multiple Membership Random Effects Models: Demonstrations, Evaluations, and Extensions

Abstract

Theoretical Framework

Increasingly, educational and social science researchers have recognized that the context to which the individuals belong might greatly influence their behaviors. One individual could belong to more than one context at the same time (such as students attending more than one elementary school). Cross-classified random effect models (CCREMs) have been developed and applied to data with cross-classified structures in various fields (i.e., education, health, transportation, social science research, etc.) (Hough, 2006; Ecob et al., 2004; Rasbash & Browne, 2001; Fielding, 2002; Goldstein & Sammons, 1997; Raudenbush, 1993).

Purpose

Given the lack of investigation of the effect of ignoring cross-classified data structures in data with binary outcomes, this study expands on the work of Meyers and Beretvas (2006), and is designed to: (1) explore the consequences of ignoring a cross-classified structure in a dataset with a dichotomous outcome; (2) examine differences due to estimation method; and (3) clarify the conditions under which there is a need to use the logistic CCREM. In addition, this study provides a demonstration of the logistic CCREM and focuses on investigating students’ reading ability and the effect of a reading intervention on the likelihood of incarcerated youths’ recidivism.

Method

First, a simulation study was conducted to investigate the impact of correctly modeling versus neglecting to model cross-classified data under a variety of conditions, using different estimation methods [specifically, penalized quasi-likelihood (PQL) vs. adaptive quadrature (AQ)]. (Table 20). Secondly, a follow-up evaluation study was conducted using data from an incarcerated youth recidivism study, where youth were nested with facilities and counties (the two cross-classified factors). We compared model estimates for the CCREM, a conventional HGLM (ignoring the facilities cross-classified factor) and an HGLM with dummy coded fixed effects identifying each facility.

Data Analysis

The models in both studies were estimated using SAS PROC GLIMMIX. For the simulation study, the values of the fixed parameter estimates and random effects variance estimates will be summarized across the 500 replications for each condition and two estimation procedures. (Note that the simulation study will be completed in time for the final paper to be presented). Four evaluation criteria were used, including: relative bias of estimates, relative bias of standard errors, RMSE, and fit indices. For the real data analysis, the coefficient estimates have been estimated (see Table 21).


Discussion

Results of the simulation study will contribute to research on estimating the HGLM and logistic CCREM. The application study explores the importance of incarcerated youth’s reading ability and the effectiveness of an in-prison reading program on their recidivism and demonstrates inferential differences that can result as a function of choice of model. The effect of the county where the youth come from and the facility where the youth are retained on incarcerated youth’s recidivism will be addressed and discussed in the final paper. In addition, explanation of the models, justification for the analysis decisions and design conditions in the simulation study as well as recommendations for applied researchers will be fully presented.

Author

• Weijia Ren, Westat