Individual Submission Summary
Share...

Direct link:

Robustness in Qualitative Comparative Analysis

Fri, August 30, 10:00 to 11:30am, Marriott, Taylor

Abstract

The analysis of necessity and sufficiency with Qualitative Comparative Analysis (QCA) requires making design decisions that are characterized by uncertainty. “uncertainty” means an empirical researcher has a range of options that are equally viable based on the available knowledge of cases, concepts and theories. These decisions involve, among others, determining the cases to include in the analysis; the choice of the set type; the specification of the calibration anchors; the type of solution that is derived from the data.

Knowing about the potential sensitivity of results to specific modeling decisions, an increasing number of QCA studies now report results from robustness tests comparing the results for different specifications. Relatedly, the methods literature has assessed the robustness of QCA to various modeling choices in recent years.

The empirical and methodological work on robustness share one crucial element. The understanding of robustness usually is that the solutions resulting from alternative modeling choices are identical to the solution following from the preferred set of specification choices (the target solution). In this perspective, a solution is either robust or not.

In our paper, we advance the debate about robustness on a conceptual and empirical level with the goal of allowing empirical researchers to make more nuanced assessments of robustness. Conceptually, we distinguish between three different levels on which we can determine robustness. The first level captures the currently common perspective that looks at entire solutions. Going beyond this view, we argue that it is important to decompose solutions into its individual disjuncts (also known as equifinality represented by the logical OR). Even if entire solutions are not robust, it might be that a disjunct – an individual condition or conjunction – is an element of every solution. This would be a relevant insight and be missed when one only looks at complete solutions.

On the third level, we can determine robustness for individual conditions, which often are INUS conditions in empirical QCA work. Again, we might observe non-robustness for solutions and disjuncts (no disjuncts occurs in all solutions), but find that an INUS condition is present in all solutions.
The distinction between solutions, their disjuncts and conditions aligns the evaluation of robustness with empirical research because confirmatory QCA studies usually contain hypotheses about individual conditions and conjunctions and not full solutions. The more often a hypothesized INUS condition or conjunction occurs in all produced solutions, the more confident we can be in the hypothesis being correct.

Based on this distinction, we replicate existing empirical and methodological with an easy-to-use R package. The R package allows empirical researchers to visualize the degree of robustness on all three levels and assess how often disjuncts or INUS conditions coincide in the same solution. The replication shows that the exclusive analysis of robustness on the level of complete solutions masks a high degree of robustness on the second and third level for individual disjuncts and INUS conditions.

Authors