Lost in Aggregation: why the level of analysis matters

• In Event: Governance Beyond the State: Social Institutions and Service Delivery

Abstract

Analyses of governance and development outcomes are usually based on country-level data, whether drawn from expert coding (e.g., QoG, WBI) or public opinion surveys (e.g., Barometers, WVS). Yet, because such analyses overlook subnational variation, ‘aggregation bias’ is likely to yield inaccurate inferences about the quality of governance and development in the country as a whole. We contend that this “micro-macro” paradox (Shaffer 2015, Nonnenmacher and Friedrichs 2013) is at the root of problems in developing a clear theoretical understanding of the links between governance and development, and thus inadvertently result in inappropriate policy responses. This paper examines the magnitude of these problems. We exploit multilevel data from a survey of more than 8,000 respondents in Malawi, representative at the local level, to consider how inferences drawn from both nationally- and locally-representative surveys differ. We trace the relationship between governance and development outcomes (e.g. property rights, security, health, education, inter alia ), that theoretically are expected to be driven by different levels of administration and evaluate systematically the robustness of our results to a variety of data aggregation strategies. We explore tipping points and variation across issue areas, investigating which conditions are important for a particular relationship and why we may miss specify them when aggregating at different levels. The paper advances theoretical understanding of the relationship between governance and development and yields insights about the design of data collection and analytical methods that can mitigate problems of aggregation bias with information collected from developing countries.

Authors

• Pierre F. Landry, NYU-Shanghai

• Ellen M. Lust, University of Gothenburg

• Sebastian Nickel