Trends and Evolution in the Conduct of Political Science Research

• In Event: Challenges in Qualitative Data Collection and Analysis

Abstract

How do political scientists produce knowledge? What data collection techniques and methods of analysis do they employ in this endeavor? How does this process differ across subfields and how has it changed over time? Previous research on how political scientists carry out their work has focused heavily on the analytic methods that they employ (Bennett et al. 2003; Blanchard et al. 2017; Levy 2007). To our knowledge, no study has surveyed the data collection / generation techniques and the analytic methods that political scientists employ, and how they combine them, when conducting their inquiries.

This project seeks to fill that gap by answering a series of linked research questions: What proportion of authors generate their own data? When authors collect or generate their own data, do they tend to use one or more techniques, and which do they tend to use? What types of data sources do they tend to tap, and what categories of political and social actors do they tend to invite to participate in their work? Is scholarship employing multiple analytic methods increasing, decreasing, or remaining static in prevalence? What types of methods do authors tend to use in tandem? Are there trends in terms of the pairing of particular data collection techniques and analytic methods? How do the answers to these questions differ across subfields and time?

In order to answer these questions, we examine the data collection and generation techniques and methods of analysis employed in a systematically selected set of research articles published in ten top political science journals between 1998 and 2018 (N=1,926). Examining articles published during this time period allows us to analyze the effect of the renaissance of qualitative methods since 2000, which we hypothesize may have acted as a catalyst for the publication of more multimethod articles. The selected time period also overlaps with the time period covered in previous research (e.g., Bennett et al. 2003).

We identified the journals to include in our study based on two sets of journal rankings, Giles and Garand (2007) and Garand et al. (2009). The top ten journals in each ranking overlapped considerably (after excluding sub-subfield journals such as Legislative Studies Quarterly). Limiting our analysis to ten journals allows us to maximize comparability with the work of Bennett et al. (2003) and capture cross-subfield breadth, while leaving us a manageable analytic task. Of the ten journals, five are general (American Political Science Review, American Journal of Political Science, British Journal of Political Science, Journal of Politics, and Perspectives on Politics), two predominantly focus on International Relations (International Organization and International Studies Quarterly), two focus on Comparative Politics (Comparative Politics and Comparative Political Studies ), and one publishes work from both International Relations and Comparative Politics (World Politics).

To be eligible for inclusion in our sample, articles must be peer-reviewed and contain substantial original empirical research; the sample excludes journal selections such as research notes and book reviews. For symposia and special issues, articles that adhered to the selection criteria were eligible for inclusion. After identifying all of the articles that met our criteria, we randomly selected a quarter to code, resulting in a sample of 1,926 journal articles. Using computer-assisted qualitative data analysis software, we code each article using three categories of codes: general information, data collection and generation techniques, and data analysis method.

We have several hypotheses and expectations based on our examination of initial data. First, over time and across subfields, we expect to find that an increasing number of scholars are generating their own data, and their own qualitative data in particular, through their research. Relatedly, we also expect to see an increase in multimethod research, especially research using both quantitative and qualitative methods, and an increase in work that is primarily quantitative employing some qualitative analytic technique in particular. Finally, we expect to see a sharp increase in more technologically advanced methods such as machine learning in the latter half of the study period.

Political science is a heterogeneous discipline: scholars use a range of data-gathering techniques and analytic methods to generate knowledge about fascinating political dynamics. Our paper should reveal important trends and evolution in the way scholars conduct their research. Its findings will have implications for how we teach political science, carry out our work, and think of ourselves as a discipline.

Authors

• Tranae Hardy, Georgetown University

• Diana Kapiszewski, Georgetown University