Bayesian Nested Analysis

• In Event: Innovations in Case Study Methods

Abstract

The development of nested analysis marks a seminal contribution to empirical social science by moving beyond the hitherto standard distinction between qualitative and quantitative research. In the original formulation, empirical researchers achieve this by systematically integrating frequentist regression analysis with the in-depth study of cases selected based on the estimates. The formulation of nested analysis was the starting point for numerous empirical applications and advancements on concept formation, case selection and its transfer to spatial regression and set-theoretic research.

In this paper, we contribute to the advancement of multimethod research and develop Bayesian nested analysis (BNA). BNA differs in four salient ways from the original, frequentist version of nested analysis (FNA) that so far has been at the center of empirical and methodological work. First, Bayesian nested analysis with at least one quantitative and qualitative stage allows researchers to formally use the knowledge gained with one method to inform the implementation of the other method in the next stage. We explain in detail how empirical researchers can use a Bayesian framework to introduce prior knowledge at the three interfaces of qualitative and quantitative methods in nested analysis: the use of varying levels of background information about a specific case or set of cases to build the first statistical model; the use of the regression results to choose cases for in-depth analysis and the actual process tracing study; the use of the process tracing insights to run an additional quantitative analysis.

Second, the regression estimates for a frequentist and Bayesian regression are the same for uninformative, flat priors, but need to be interpreted differently. We demonstrate that the Bayesian interpretation is more appropriate for the research goals of a nested analysis. Empirical researchers should choose cases based on the range of probable causal effects, "probable" given the parameters of the Bayesian regression, and not the probability of sampled data over an infinite number of repeated samples given a null hypothesis.

Third, the use of informative priors for the initial regression analysis creates a difference between the frequentist estimates and the Bayesian posterior estimates beyond their interpretation. Different estimates are likely to lead to the choice of different cases for process tracing because the residuals of the same cases differ between FNA and BNA.

Fourth, the qualitative literature developed a Bayesian framework emphasizing the importance of the weight of observations captured in the likelihood. In contrast to frequentist regression, BNA allows empirical researchers to operate within a coherent framework and make integrated Bayesian quantitative and qualitative inferences. We present simulations and replications of published articles using nested analysis to demonstrate the four advantages of BNA.

Authors

• Ingo Rohlfing, Universität zu Köln

• Lion Behrens, University of Mannheim