A Qualitative Approach to Moderation Analysis

• In Event: Causality, Processes, and Mechanisms

Abstract

The present paper advances a qualitative approach to moderation analysis in experimental political science. In the last decades, experimenters have developed different empirical strategies for dealing with external validity issues or the limited generalizability of their findings from one context to another. Concerns about external validity are currently a constitutive feature of the most recent wave of experimental research (Druckman and Green 2021). Among these strategies, moderation analyses focus on “second-order causal inference problems” – they try to identify moderators (also referred to as “background conditions,” “support factors,” or
“interactive factors”) or variables (Z) that affect the effect of the treatment (X) on the outcome (Y). Scholars typically perform moderation analyses by applying various quantitative methods, from simple regression analysis to sophisticated ensembles of machine learning algorithms that identify causal heterogeneity or heterogeneous-treatment effects. However, nothing in the definition of moderation analysis prevents applying qualitative methods. On the contrary, statistical methods are not always appropriate for moderation analysis because of power issues related to a small sample of experiments and their daunting assumption.

This article offers an alternative way of conducting moderation analysis based on qualitative methods. It argues that process-tracing tests, set-theoretical tools, Boolean algebra, and Qualitative Comparative Analysis (QCA) helps identify moderators even in the presence of a small sample of experiments. The article develops valuable criteria for applying these qualitative methods by considering existing experimental research.

Author

• Daniel Encinas, Northwestern University