The Puzzle of Treaty Ratification: A Causal Variable Importance Analysis

• In Event: Research in Causal Inference and Applications

Abstract

International human rights law is crafted with the explicit goal of protecting and promoting universal human rights. It establishes substantive obligations for states parties and designs procedural mechanisms to implement and monitor the implementation of these obligations. Given that states could be well aware of the potential costs and consequences of treaty ratification, a natural question arises as to why more and more countries nonetheless ratify human rights treaties that are designed precisely to limit their freedom in treating their own citizens. This is simple, yet vexing, question. The literature is still deeply divided. Many theories have been proposed only to get theoretically disputed or empirically refuted, leaving the question of human rights treaty ratification as an enduring puzzle.

While solving this puzzle is undoubtedly difficult, existing empirical inquiries reveal a couple of major limitations. First, they mostly rely on statistical significance of regression model coefficients to infer about variable importance. These models almost always make parametric assumptions such as linearity and additivity. Usually no justifications are provided as to why a linear functional form, for example, is appropriate or likely reflects the true causal dependence of the outcome on its predictors. Since we usually do not know a priori the underlying data-generating process, particularly for complex social and political phenomena, a conveniently specified statistical model is likely a misspecified one.

The second limitation is that virtually every study presupposes a causal query about the determinants of treaty ratification. Yet, none has openly embraced a causal language and causal inference framework within which to formulate its question and conduct analysis. This inattention to causal identification has unfortunate implications. For one, scholars may not appropriately separate causal identification from statistical estimation. Endogeneity issue, for example, is often viewed as a statistical problem rather than an identification issue. Researchers therefore often mistakenly use estimation techniques such as propensity score matching for the task of identification. As a result, causal identification in an observational setting often is not addressed explicitly. For another, researchers often fail to employ powerful tools of causal inference such as the backdoor criterion to guide their covariate selection.

We address this conundrum of treaty ratification by employing Pearl's causal inference framework and ensemble machine learning techniques to conduct a causal variable importance analysis. The goal is to learn about the extent to which fixing the value of a predictor variable will change the distribution of state ratification of three UN core human rights treaties, including the International Covenant on Civil and Political Rights, the Convention on the Elimination of All Forms of Discrimination against Women, and the Convention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment. We, first, formulate our substantive question of variable importance as the magnitude of causal effects. We then use the rules of causal inference to identify these causal effects as statistical parameters that can be estimated from observational data. We use either the state-of-the-art machine learning method of extreme gradient boosting or the ensemble technique of Super Learner to compute parameter estimates. Finally, we interpret the findings in substantively relevant language and draw out their implications.

The contributions of this paper are three-fold. First, we subject multiple existing theories to a different kind of empirical testing that does not rely on parametric regression models. Our machine learning-based causal inference test is more transparent in its causal identification and less dependent on parametric assumptions. These two features of identification transparency and modeling flexibility are missing in current empirical inquiries.

Second, our analysis provides new substantive insights into the causal determinants of treaty ratification. Previous research has analyzed predictors of state commitment to many universal treaties. Others have applied random forest technique to examine non-linear relationships between predictor variables and human rights outcome. Our investigation improves upon the former by using machine learning in lieu of parametric models and upon the latter by endowing our findings with a causal interpretation.

Finally, this paper introduces a readily applicable template of analysis that combines causal inference and machine learning for political science research. Given that almost every phenomenon worth examining in social sciences is multicausal, the need to evaluate the relative causal importance of predictor variables is crucial not just for our understanding but also for proposing effective interventions to induce desired effects.

Author

• Sinh Nguyen, Purdue University