The Impact of Poverty on U.S. PISA (Programme for International Student Assessment) Achievement: A Propensity Score Matching Approach

• In Event: Studying Educational Inequalities From an International Perspective

Abstract

Objectives
A well-established link exists between poverty and lower educational achievement (Campbell & Ramey, 1994; Entwisle & Alexander, 1992; Kao & Thompson, 2003; van der Klaauw, 2008). Further, the UNICEF reported that 32.2% of US children were living in poverty in 2012, up 2% from 2008 – among the highest in economically developed countries (UNICEF Office of Research, 2014, p. 10). As wealth gaps continue to widen and more children face the stark reality of living in poverty, the importance of understanding the effects of material deprivation grows. To that end, we use the most recent PISA data to estimate the causal effect of poverty on achievement in the US.

Theoretical framework
Several studies linked race and ethnicity (e.g., Kao & Thompson, 2003) and immigrant status (e.g., Suárez-Orozco, Bang, & Onaga, 2010) to achievement gaps; however, these gaps can be at least partially explained by systematic differences in SES (Kao & Thompson, 2003; Pong, Dronkers, & Hampden-Thompson, 2003). And although at least one study used a longitudinal design to attribute achievement gaps to SES differences (Entwisle & Alexander, 1992), these studies generally do not estimate the causal effect of poverty on education, offering only correlational inferences. As such, we use a propensity score matching approach to identify a causal description of poverty-based achievement gaps.

Methods
The basic theory behind propensity score matching is to approximate the counterfactual by identifying a control group (not in poverty) that is as similar as possible to the treatment group (in poverty) through a matching procedure. The average outcome for the control group, if well matched, is argued to approximate what we could have expected if the treatment group had not been treated (Rosenbaum & Rubin, 1983). We classify students in poverty as those in the lowest 32% of the PISA SES measure (based on the UNICEF report). We then use several nearest-neighbor matching algorithms to match poverty and non-poverty students. Achievement differences between these two groups can be regarded as the unbiased treatment effect of poverty (Baser, 2006; Caliendo & Kopeinig, 2008; McEwan, 2010; Steiner & Cook, 2013). We evaluate matching quality and conduct a sensitivity analysis to ascertain whether hidden bias threatens the validity of our results.

Data
We use student background questionnaire data and mathematics achievement from the 2012 PISA cycle to estimate poverty-based achievement gaps in the US. Full details on the measures used to match students and the estimation procedures are included in the full manuscript.

Results
We found good balance on the matching covariates and that our results are relatively robust to unobserved hidden bias. Further, we estimate the poverty gap to be 26.75 points on the PISA scale.

Scientific significance
The specter of educational and, by extension, economic failure looms large in US policy conversations. Controlling for many other factors and in line with other studies our finding suggest that eliminating the effect of poverty would raise achievement above the OECD average, implying that more citizens would have the skills to fully participate in a global society.

Authors

• David Joseph Rutkowski, University of Oslo

• Leslie Rutkowski, University of Oslo

• Justin Wild, Indiana University