A Within-Study Comparison of Experimental and Observational Estimates of Income Effects on Child Development and Maternal Well-Being

• In Event: Projecting Intervention Impacts in Developmental and Educational Studies

Abstract

Research assessing impacts of income on child development often relies on analyses of nonexperimental data that can be biased by the omission of other sources of disadvantage that have effects on both income and child development. However, consistent evidence from experimental and quasi-experimental research indicates that income has a causal influence on how individuals develop during middle childhood, adolescence, and early adulthood. Therefore regression-adjusted correlations between income and children’s later development may in part reflect a causal mechanism at work in early childhood. The correspondence between the estimated income gradients from nonexperimental cross-sectional and longitudinal data on child development and maternal well-being outcomes has yet to be compared systematically with impacts from experimental manipulations of income.

We analyze data collected in the first randomized controlled trial (RCT) study in the U.S. to measure the impact of an unconditional child allowance on child development and maternal well-being (Magnuson, Noble, Duncan, Fox, Gennetian, Yoshikawa, 2020). Data were drawn from 1,000 mothers with newborn infants who were randomly assigned to receive a monthly cash gift of either $20 (control group, n=600) or $333 (treatment group, n=400) each month for the first 12 months of the child’s life. Thus, the compensation difference between low-cash and high-cash gift groups is a $3,760 higher annual income. We use the publicly available data from the first wave of data collected during the 1st year of the child’s life. Further descriptions of participant enrollment and study design are available in shown in Noble, Magnuson, Gennetian, et al. (2021).

We use a within-study comparison (Shadish, Cook, & Wong, 2008) to determine whether income gradients approximate experimental impacts on child development and maternal well-being outcomes in magnitude and direction. We first use the data from participants in the treatment and control groups to estimate the intent-to-treat (ITT) estimand (equation 1β_ex) using multivariate regressions with baseline covariates for each outcome.:

〖(1) Outcome〗_i=β_0+ β_ex 〖Trt〗_i+ β_2 〖Cov〗_i+e_i

Then, we isolate data from the participants in the control group to estimate nonexperimental income gradients (equation 2 β_ne) by regressing each outcome on income while controlling for the same baseline covariates included in the ITT model.

〖(2) Outcome〗_i=β_0+ β_ne 〖Income〗_i+ β_2 〖Cov〗_i+e_i

We compare estimate correspondence for all the measured outcomes we plotted the control group standardized estimates in Figures 1 and 2. In Figure 1, we plot the linear regression estimates without covariates (left) and with covariates (right). The nonexperimental linear regression estimate magnitude is reflected on the y-axis and the experimental impacts is reflected in the x-axis. If the estimates perfectly corresponded, they would fall along the 45-degree dotted line. Figure 2 compares the nonexperimental log income gradients rescaled to reflect a $1,000 annual income increase. In summary, we find that experimental impacts are much more variable than the nonexperimental estimates. Nonexperimental income gradients are generally insignificant, and they tend to be smaller in magnitude and less variable than experimental income impacts. We discuss the implications of this work for the challenge of attempting to craft evidence-based policy based on experimental and non-experimental data.

Authors

• Daniela Alvarez-Vargas, University of California - Irvine
  Presenting Author

• Drew Bailey, University of California - Irvine

• Greg J Duncan, University of California, Irvine