Credible or Confounded? Revealing & Reporting the Fragility of Causal Estimates

• In Event: Causal Inference: Mediation, Experiments, and Sensitivity

Abstract

Social scientists often find themselves attempting to learn the effect of some factor on an outcome, but unable to eliminate all potential confounders so as to make regression or other estimates credibly reflect causal quantities. Rather than either abandoning hope or overlooking these problems, we demonstrate a sensitivity-centered approach to making progress in such circumstances, using a recently developed suite of sensitivity analyses (Cinelli and Hazlett, 2018). These tools offer interpretable metrics for revealing the fragility of a causal claim that can be routinely reported or extracted from regression output. These tools provide precise accounts of the confounders that would alter our conclusions, allowing readers to decide if a finding is difficult to defend, arguably defensible, or somewhere in between. We apply these tools to study the highly polarized 2016 referendum for peace with the FARC. Two major influences on voting behavior have been proposed: (1) exposure to violence, and (2) political affiliation with President Santos. Conventional regression analyses find "statistically and substantively significant" estimated effects for both factors. However, we show that the effect of exposure to violence is extremely fragile and can be overturned by even very weak confounders. By comparison, the estimated effect of political affiliation requires much stronger confounding to overturn. These results illustrate how focusing on the sensitivity of an estimate to unobserved confounding can generate insights that advance inquiry in cases where "perfect identification" is out of reach.

Authors

• Chad Hazlett

• Francesca Parente, University of California, Los Angeles