Choice of Sampling Weight for Longitudinal Modeling of Panel Data

• In Event: Leveraging Federal Data in Education Research: Engaging With Administrative and Complex Sample Data

Abstract

National longitudinal panel surveys in education, such as the Early Childhood Longitudinal Study Kindergarten (ECLS-K: Tourangeau et al., 2009), face the problem of respondent attrition. On public-release data files for these studies, sets of panel weights are provided that allow the analyst to appropriately weight the observations in the study to account for such attrition. Several sets of weights are provided depending on which waves of data are intended to be used in the analysis. However, these weight adjustments are based typically on sampling design information and data collected in the base year; information obtained from early waves in the survey program is not typically utilized in the adjustment model which may lead to bias in estimates if attrition and the outcome are related to these data. Another method to address potential bias from non-response is full information maximum likelihood (FIML; Arbuckle, 1996). This method makes use of responses obtained during all data collection waves. In this presentation, we describe commonly-used approaches to sampling weight adjustment for non-response and attrition, including cell adjustment, response propensity models, poststratification and raking (Kalton & Kasprzyk, 1986; Little, 1986).

We present arguments for using FIML with auxiliary variables when undertaking longitudinal analyses. Using data from ECLS-K we demonstrate the differences in estimates from analyses when non-response adjusted panel weights are used compared to FIML utilizing the base year sampling weights only with auxiliary information. Specifically, we work with the following research question: Are there differences among children in the growth characteristics in math performance from kindergarten to 8th grade and, if so, are these characteristics a function of gender? We fit a Gompertz function to a subset of the data (Asian/Pacific Islander students who attend private schools) and compare results using the panel data (the individuals who responded at every wave, N=97) and adjusted panel weights with results from utilizing all students who were in the base wave of data collection (N=274) using FIML with auxiliary variables and the base wave weight.

The fitted curve generated from the parameter estimates resulting from FIML with the base wave weights are slightly lower than the curve resulting from the estimates from the panel weighted analysis. In addition, with the panel weighted analysis, we would reach the conclusion that males score higher than females at the asymptote (potential performance level), a conclusion we would not reach using FIML. Because these are empirical data, we cannot be sure which estimates are unbiased, if any. However, these discrepancies suggest that bias may exist when using panel weights and ignoring the attrition that is informative for a given outcome variable. The inclusion of additional relevant auxiliary information may lead to more appropriate inference. We also demonstrate, via simulation, the effects of bias in growth parameter estimates using panel weights and FIML when informative auxiliary variables (that define missingness) are not utilized. Specifically, we vary the degree to which auxiliary information is correlated with the outcome variable as well as response status and unit non-response rate.

Authors

• Laura M. Stapleton, University of Maryland - College Park

• Jeffrey R. Harring, University of Maryland

• Daniel Y Lee, University of Maryland - College Park