Conformal Prediction of Armed Conflict

• In Event: What We Can Know about the Future of War and Its Humanitarian Impacts?

Abstract

Recent advances in methodology and data availability have propelled forecasting of political variables such as counts of fatalities in armed conflict into the mainstream of conflict research. While these advances are important, the forecasts themselves are still mainly expressed as point predictions, which is problematic for a variety of reasons. First, point predictions may be reasonably accurate but are surely wrong, and so without uncertainty quantification of associated forecasts, the utility of point predictions is markedly reduced. Second, many of the phenomena studied in conflict forecasting are rare events (resulting in zero inflated outcomes) having potentially extreme-valued tails, rendering point predictions to be exceedingly misleading. Third, point predictions without uncertainty measures are poor tools for communicating the results. Even more than a best guess (the point prediction), policy makers are typically interested in quantifying the risk associated with a given phenomenon; for instance the risk of high levels of violence versus low or no levels of violence. In this paper, we aim to introduce and demonstrate a general purpose, non-parametric approach to uncertainty quantification---called conformal prediction (CP, Vovk and Shafer 2005, Shafer and Vovk 2008) -- that can be implemented on top of virtually any model that produces point predictions. Assuming, for example, that residuals of the point prediction forecasts for a given underlying model are exchangeable (or iid), CP sets can be constructed to yield a confidence region for a point prediction for any user-specified level of significance. Whereas other methods exist that can provide uncertainly quantification for point predictions (e.g., credible sets from a Bayesian model), CP sets are mathematically guaranteed to be calibrated to their nominal (user-specified) level of coverage in repeated samples (i.e., non-asymptotic control over type 1 errors). We investigate an application of constructing CP intervals for quantifying the uncertainty associated with point predictions from the Violence Early Warning System (ViEWS) fatalities forecasting model(s). To construct CP intervals, we consider a variety of measures of dissimilarity or non-conformity (e.g., squared or absolute prediction residuals from forecast models), and we evaluate their empirical coverage and (average) width. We compare these CP intervals to prediction intervals created from other methods such as parametric and bootstrapping approaches, and we evaluate their resulting Continuous Rank Probability Score (CRPS) and Mean Scaled Interval Score (MSIS).

Authors

• David Randahl, Department of Peace and Conflict Research, Uppsala University

• Jonathan P Williams, NC State University

• Håvard Hegre, Peace Research Institute Oslo