Three-Level Models for Partially Nested Data Structures

• In Event: Promising Scholarship in Education: Dissertation Fellows and Their Research

Abstract

An important goal in much educational research is to determine the effectiveness of intervention programs, especially those designed to improve achievement outcomes for students in disadvantaged schools and neighborhoods. The designs of studies of program effectiveness are often quite complex and involve complicated data structures. In many cases, analytic techniques have not been developed to address such complicated data, and researchers are left to apply existing analytic methods (that were developed to address simpler designs) with questions emerging about the validity of the results of the analyses. The purposes of this study were (1) to extend previously developed analytic methods so that they can be applied to more complicated (and realistic) designs, and (2) to compare the accuracy of conclusions about program efficacy using various analytic approaches.
Partially nested data structures occur when some units (typically individuals) are nested within groups while others are not nested, as can arise in some study designs, such as individually-randomized control trials. An adjustment to the standard multilevel model for 2-level data structures to accommodate partially nested data was proposed by Bauer, Sterba, and Hallfors, where individuals at Level 1 are either nested within treatment groups or are independent at Level 2. To accommodate partially nested data structures, the multilevel model is specified with a fixed intercept and random slope at the treatment group level. This parameterization yields a model that treats control arm individuals as non-nested at the treatment group level, while estimating the average treatment effect and variability in the size of the treatment effect across treatment groups for the treatment arm. The model also estimates separate Level 1 residuals for control and treatment arms. The focus of the current body of work was on extending this 2-level model in two ways.
The first extension involved incorporating a higher level of nesting, using an example where individuals within schools were randomized into treatment and control arms. Individuals at Level 1 were either nested within treatment groups or were non-nested (control arm) at Level 2, but all individuals were nested within the schools from which they were sampled at Level 3. Varying factors such as sample size, ICC at Levels 2 and 3, and the degree of heteroscedasticity between control and treatment arms, a simulation study examined the performance of this 3-level model. Specific hypotheses were tested with regard to omitting the school level of nesting and the advantages of specifying a heteroscedastic model over a more parsimonious homoscedastic model when systematic differences in variance between control and treatment arms are expected. Further, the advantages of the 3-level model in terms of including school-related predictors were examined.

Author

• Jessica Tessler, California State University - Fullerton