The Uncounted Dead: Aerial Bombardment and Civilian Fatalities in Conflict Data

• In Event: Critical Challenges and Solutions towards (Re)Constructing Conflict Datasets

Abstract

Quantitative research on political violence relies extensively on “off-the-shelf” datasets, such as those compiled by the Uppsala Conflict Data Program (UCDP) and Correlates of War (COW) project. While used for both descriptive and causal inference, these datasets are fundamentally descriptive in nature, as they purport to identify cases of relevant phenomena, such as armed conflict or violence against civilians, and/or measure their intensity, frequency, and other salient characteristics. Studies using these datasets implicitly assume that they accurately describe the phenomena under analysis, and researchers often employ these data unquestioningly, without considering or addressing possible concerns about their accuracy, validity, or completeness. However, even a cursory overview of widely cited political violence datasets suggests that “off-the-shelf” data are often incomplete or otherwise inaccurate; for instance, the UCDP One-Sided Violence (OSV) dataset (Eck and Hultman 2007), which purports to measure civilian killings, records no entries for Burundi (1993), during which there were well-documented mass killings that, according to a UN Commission of Inquiry (2002), included acts of genocide. More broadly, recent critiques have identified biases in the OSV and other conflict datasets resulting from their reliance on newswire, other media, and NGO or IGO reports, leading to missing cases and significant undercounting (Dawkins 2020; Farriss et al. 2020; Krüger and Nordås 2020). Building upon these critiques, we propose to develop a broader catalog and typology of missingness and undercounting in “off-the-shelf” data on political violence, focusing on the UCDP OSV. Based on analysis of the underlying source data used to construct UCDP datasets, including the OSV, and comparisons with non-UCDP datasets, we initially identify three general types of civilian killings missing from or undercounted in the OSV: 1) civilian fatalities recorded in the source data but classified as “battle-related” or “unknown” and therefore omitted from the OSV (e.g. Sri Lanka, 2008-2009); 2) civilian deaths recorded in the source data but definitionally excluded from the OSV because they occurred in communal or other forms of “non-state” conflict not directly involving state forces (e.g. Mexico, 2010-present); and 3) civilian deaths not recorded in the UCDP source data (e.g. Burundi, 1993). We then propose and examine several possible explanations for missingness and undercounting of each of these types, including technology of violence employed (e.g. airstrikes and long-range bombardment), actor and conflict types, and international media and human rights NGO presence, inter alia. While our analysis focuses specifically on the UCDP OSV and civilian killings, the explanations for missingness and undercounting that we examine are potentially more broadly applicable to other forms of political violence; at the same time, our analysis also points to some of the practical and methodological challenges in identifying missing cases, particularly when comparing datasets that rely on similar underlying sources.

Authors

• M.P. Broache, University of North Carolina at Greensboro

• Kate Cronin-Furman, University College London

• Milli Lake, London School of Economics

• Agnes Yu, London School of Economics