Addressing Endogeneity Selection Bias in a Non-Experimental Study of Academic Achievement

• In Event: 3-238 - New (or Under Appreciated) Statistical Methods for Probing Causal Hypotheses in Developmental Research

Integrative Statement

The non-random assignment of children to treatment conditions in observational studies often precludes credible causal inference (Hernan & Robins, 2013). This is because of the potential for unmeasured factors associated with selection into treatment to bias results. Technically speaking, the central issue in causal inference research is the assumption that the variation in the causal variable (e.g., treatment) is exogenous and the tenability of this assumption under different designs or methodological approaches (Gerring, 2011; Pearl, 2009). Randomized experiments (otherwise known as RCTs) ensure exogeneity because the assignment mechanism allocates study elements to the treatment and control conditions randomly and thus does not depend on the outcome variable being studied, or on any variables related to the outcome variable (Stock & Watson, 2014). However, in the case of research scenarios with non-random assignment, like the one presented here, treatment assignment may no longer be independent of the outcome variable or of its correlates.
In this paper, the researcher applies two novel approaches to dealing with endogeneity bias to uncover the causal impact of an “integrated student support” intervention. The intervention, City Connects (CCNX), was developed to mitigate out-of-school barriers to learning through a systematic and coordinated process within elementary schools. However, because schools have not been randomly selected to participate in the CCNX intervention and because children do not randomly choose which school they attend within their district’s school system, any study examining the effects of the CCNX intervention model will need to address endogeneity selection bias. Traditionally, developmental researchers have addressed endogeneity selection bias by means of covariate adjustment and propensity score methods; however, these methods are problematic in that they invoke the underlying assumption that all heterogeneity between the treatment groups can be captured by observed variables (Baser, 2006). Given school choice is determined by a myriad of non-random factors (e.g., parental education, neighborhood) and all these variables cannot be made available to the researcher, such an assumption seems unreasonable.
The two novel approaches presented in this paper deal with both observed and unobserved confounders. The first approach, Deferred Acceptance (DA) Propensity Score Instrumental Variable (IV), relies on simulating random offers from a deferred acceptance algorithm used by a large urban school district to decide school placement in a lottery process. Secondly, an instrument-free approach, known as the Gaussian Copula OLS regression, is used to estimate the causal impact of the intervention. The researcher will then discuss the intervention causal impact, the implementation and benefits of these new approaches, and give recommendations for making stronger causal inferences using these methods. Lastly, an R package developed by the presenting author to allow researchers to easily apply the simulated DA Propensity Score IV approach to their own substantive research will be highlighted and shared.

Authors

• Jordan Lawson, Boston College
  Presenting Author

• Laura O'Dwyer, Boston College
  Non-Presenting Author

• Mary Walsh, Boston College
  Non-Presenting Author