Examining the Validity and Reliability of Alternative Student-Level Socioeconomic Status Measures

• In Event: 41.066 - The Development and Applications of Alternative Student Socioeconomic Status Measures

Abstract

The student eligibility for the National School Lunch Program (NSLP) has often been used as a student-level SES construct in educational research. However, the measure of NSLP eligibility has become less valid and reliable over time due to reasons such as the adoption of school- and jurisdiction-level eligibility and that eligible students who tend not to apply increases with grade level. Based on one of the key recommendations from the NAEP SES Expert Panel (2012) that researchers should focus on developing a SES composite measure based on the “Big 3” indicators (family income, parental educational attainment, and parental occupational status) rather than relying on single proxy measure, the author of this paper explored various NAEP SES-related student questionnaire items and developed a student-level SES composite measure from the 2009 NAEP Grade 12 dataset.
Building upon author’s previous research, this paper examines the validity and reliability of the student-level SES composite measure on the 2011 NAEP Grade 8 dataset and adapts for the 2011 NAEP Grade 4 dataset. Comparisons of the SES composite measures (with and without NSLP eligibility in the construct), variables of the parental educational attainment (for grade 8), index of the home resource availability and home possessions, and student eligibility of NSLP are conducted to examine the impact of various SES measures on student achievement (mathematics and science for grade 8; mathematics and reading for grade 4). Student demographic variables (i.e. gender and race/ethnicity) are examined to provide explanations for possible achievement gap issues and impact of student SES. Correlational analyses and internal consistency are also conducted for examining the validity and reliability of the SES composite measures.
Findings from this study illustrate the validity and reliability of the SES composite measures in comparison to other NAEP proxy questionnaire variables such as parental educational level, the availability of home resources, and student eligibility of NSLP. In addition, findings from this study also demonstrate the effectiveness of the SES composite measures in capturing variability of the socioeconomic factors and their impact on student achievement among various student demographic groups. Furthermore, the continuous composite measure of student-level SES offers additional insight into the differences observed in student achievement scores and across achievement levels by capturing the variability of students’ family income level within the sample and explain the potential large SES differences within the non-eligible category than examining the impact of student SES using the NSLP eligibility alone.
Although results from this study in comparing the student NSLP eligibility, various student SES-related variables, and the continuous student-level SES composite measures suggest that NSLP eligibility may be sufficient for group comparisons in exploratory analysis, but the SES composite measures provide a more comprehensive picture. In addition, the continuous student-level SES composite measures offers further information into the impact of SES on student achievement than using student NSLP eligibility alone. Such phenomenon is more apparent when using the composite measure than only using student NSLP eligibility.

Author

• Judy H. Tang, Westat