Improving Estimates of Ethnic Diversity Using Transparent Methods

• In Event: Identity Politics: Race, Ethnicity, and Nationalism

Abstract

Existing estimates of ethnic, religious, and linguistic diversity are correlated cross-sectionally with a number of socio-political and economic outcomes including development, conflict, and social capital. Close examination of these data raises validity concerns: few are based on high-quality official statistics, the majority coming from questionable secondary sources. Further, criteria for group inclusion (i.e., ontologies) are opaque and inconsistently applied. Even where they appear accurate, data are static and aggregated at the country level, although they are often used to explain time-varying and spatially disaggregated outcomes. Ontologies in extant datasets are also incompatible, making comparison and integration difficult. This project improves existing measures by applying machine learning methods to compare 13.7 million responses across 180 countries with a new database of census results. An algorithm identifies survey design features that maximize accuracy, defining a compensatory weighting scheme across these features. The result is a set of survey-based demographic estimates with improved validity, even for countries lacking reliable census data. This method of triangulating surveys and official statistics is generalizable to research areas that use either source and can also inform improved survey design. The project also develops tools for linking surveys, censuses, and existing datasets based on explicit and transparent decision rules to facilitate their comparison and integration. The updated estimates of ethnic diversity challenge our understanding of how diversity impacts and is impacted by politics.

Author

• Avital Livny, University of Illinois, Urbana-Champaign