Bayesian Type Verification: Verifying Typologies with Logical Bayesianism

• In Event: Frontiers in Qualitative Methods

Abstract

Typologies and process tracing are two pillars of qualitative methodology used to answer big questions in interesting ways. Both are sometimes challenged by non-practioners as nothing more than intuition and conjecture wrapped up in seductive yet ill-defined terms. While the tone of these challenges may be based in methodological rivalry, the underlying critique highlights that the process and results of process tracing and typology generation can be opaque to non-practioners. In response to these challenges, rigorous process tracers have proposed three formalizations of process tracing based on set theory, directed acyclic graphs (DAGs), and logical Bayesianism while rigorous typological theorists have formally specified inductive and deductive methods to generate typologies. As with all formalizations, both the strengths and limitations of these methods were brought to light through the formalization.

The elementary method of process tracing is taught using the metaphor of Sherlock Holmes and uses colloquial names for four process tracing tests (straw in the wind, hoop, smoking gun, and doubly decisive). The set theoretic and DAG approaches both formalize this vivid imagery into graphical representations while the logical Bayesian approach formalizes the informativeness of the evidence into numerical representations of human sensory perception. Bayesian process tracing best disciplines the analytic process and provides clarity to the intermediate and final results in terms consistent with human perception and the real number mathematics which are a part of general rather than specialist education.

One of the key requirements of the Bayesian approach is that it requires a mutually exclusive and exhaustive (MEE) specification of the rival hypotheses in order to properly adjudicate between rival hypotheses because it places odds ratios only on pairs of hypotheses. While MEE hypotheses are universally good research practice, most methods and the other process tracing formalization do not require MEE hypotheses. While leading BayesPT scholars have pointed out that it is always possible to rephrase a given set of hypotheses into an MEE specification, most hypoteses are not initially specified as MEE. Although the respecification is logically straightforward, the extra work and complexity of the final set of hypotheses has led some critics to question the value MEE and BayesPT more broadly due to the "cost of entry." Thus, while not MEE is not a logical limitation of BayesPT, it can be a practical one which we can alleviate with deductive typological theory.

When generating typological theories, scholars can either inductively generalize from empirical specifics or deductively combine the scores of generalized concepts to generate a set of types. Inductive typological theory runs the risk of missing logically possible combinations which have not (yet) empirically occurred. Deductive typological theory runs the risk of over specifying possible combinations which are empirically uninteresting or irrelevant. While the final typologies which are published generally involve both of these fundamental theory generating processes to some degree, the way in which they are used is often vague in final research reports.

In this paper, I demonstrate how the strengths of deductive typological theory can alleviate the limitations of logical Bayesian process tracing and vice versa. By definition, deductive typological theory completely maps a typological property space by constructing a mutually exclusive and exhaustive list of types based on combinations of constitutive variables. While Bayesian process tracing generally solves the requirement for exhaustive hypotheses by reasoned assumption, the process of deductive typological theory allows us to meet the requirement by design. Because Bayesian process tracing disciplines and clarifies how we are using case-specific knowledge and expert analysis, it allows us to transparently verify deductive typologies with inductive empirical knowledge. Together, deductive typological theorizing generates a set of mutually exclusive and exhaustive types which are then tested against empirical cases using logical Bayesianism to say how confident we can be that a particular case fits a given type. The results of this process can then be used inductively to refine the typology and applied iteratively to generate a final typology which balances analytic usefulness against empirical correspondence. Because the types developed by deductive typological theory need not be causal, I call this method Bayesian Type Verification (BayesTV) to distinguish it from the causal claims of Bayesian process tracing (BayesPT).

Author

• Konrad Posch, University of California, Berkeley