Tracing Mechanisms in Practice

• In Event: Separatism and Democratization: Using Ethnography to Identify Mechanisms

Abstract

How can we empirically trace the operation of causal mechanisms in case studies? Current social science methods offer us few methodological tools to open up empirically the ‘black-box’ of causal mechanisms to understand how a cause (or set of causes) produces an outcome. Most common are counterfactual-based designs that transform the study of mechanisms into lower-level ‘mini-experiments’ that do not enable us to make evidence-based inferences that answer research questions like ‘how does issue framing actually work?’ or ‘how does the incumbency advantage actually enable re-election?’. Even recent advances that claim to enable analysis of process only provide guidance for working with input/output type observables, for example by assessing whether a policy entrepreneur with a particular idea was present during the negotiations and whether the particular idea manifests itself in the final outcome (e.g. Jacobs, 2014). Yet this is merely circumstantial evidence that tells us nothing about the actual process; i.e. what the entrepreneur was actually doing and how her activities were received in the decision-making group in ways that might enable us to infer that the activities actually had a causal impact in the case in hand.

This paper attempts to rectify this problem by developing a practical methodology for how we can empirically measure mechanistic evidence using process-tracing. Mechanistic evidence is within-case observational evidence of the operation of activities linking parts together, enabling inferences to be made about how a causal mechanism works (Illari, 2011; Russo and Williamson, 2007). Instead of asking whether things could have been different, causal inferences using mechanistic evidence are enabled by observing the traces of activities that transfer causal forces through a causal process. What we then want to capture empirically are the traces left by the data-generating processes of the activities involved in each part of the mechanism in a particular case.

While there have also been important developments relating to what types of evidence we are working with when attempting to trace mechanisms in the social sciences, in particular regarding ideas about 'causal process observations' (CPO's) that move us in the direction of thinking about other forms of evidence than difference-making (Collier, Brady and Seawright, 2010; Mahoney, 2012; Blatter and Haverland, 2012), there is still almost no guidance in the existing literature on CPO’s for what types of empirical material can act as evidence (much of the literature is still trapped in a counterfactual-mode of thinking), nor why it can act as evidence. The value-added of this paper is that it develops a clear methodological framework for working with mechanistic evidence in practice, enabling scholars to study the actual operation of causal processes as they play out in real world cases.

To do this, we develop a four-step framework for operationalizing the observable manifestations of the activities associated with the operation of parts of causal mechanisms in case studies. The first step is to elucidate the theorized activities for each part of the mechanism in as much detail as possible, enabling the analyst to think more systematically about what types of mechanistic evidence might be left in the empirical record. The next step is gaining an overview of the actual empirical record to gain an idea of what types of mechanistic evidence might have been left (pattern, trace, sequence or account), here thinking more like a historian when mapping the empirical terrain of a particular case. The third step is then to game out what empirical traces the operation of these activities might have left in a case, being as open and creative as possible about the observables the data-generating process of activities. The fourth step involves putting forward justifications for what inferences finding/not finding particular pieces of mechanistic evidence enable about the underlying causal mechanism being analyzed. Here we draw on recent developments in Bayesian logic in process-tracing relating to what types of inferences are possible. We utilize Ziblatt’s 2009 theory on electoral fraud as a practical example, first reconstructing the theory to flag the activities of each part of the process, then develop ideas about what types of observables these activities might actually have left that could be studied empirically.

Author

• Rasmus Brun Pedersen, University of Aarhus