Qualitative Field Research, Research Quality & Research Transparency

Session Submission Type: Full Paper Panel

Session Description

Political scientists across research traditions generally recognize the importance of research communities developing consensual norms to judge research quality, even without embracing the possibility or appropriateness of universal criteria and standards. In the last two decades, deliberations on data access and transparency in the production and analysis of qualitative data have been at the center of research quality debates among qualitative and mixed-methods researchers. The Data Access and Research Transparency (DART) within the American Political Science Association (APSA), the Qualitative Data Repository (QDR) hosted by the Center for Qualitative and Multi-Method Inquiry at Syracuse University, and initiatives such as Active Citation and Annotation for Transparent Inquiry (ATI) are leading examples of a process that is currently unfolding and that still generates controversies.

This panel aims to contribute to the enterprise of developing consensual evaluative norms for qualitative methods by focusing on issues of transparency in the production, analysis and reporting of data collected via field research that involves interaction with human subjects. Field researchers use a variety of data collection/production techniques that involve human subjects, from direct observation of individuals in their daily routines, interviews and focus groups to map drawing and memory workshops and negotiating access to unique archival material. Discussing research transparency in the production, analysis and reporting of this type of data within the context of field research involving human subjects becomes all the more important as it arguably holds the most potential to raise ethical concerns.

This panel brings together a set of papers that identify specific challenges in collecting/producing, analyzing, and reporting different types of qualitative data, and propose possible concrete solutions. It hopes that, by taking stock of both these challenges and the strategies to overcome them, could help improve research transparency by identifying sets of norms and practices that scholars can adopt when using specific qualitative techniques. With this focus, this panel aims to move current debates on research transparency forward by discussing how one can make one’s work transparent, rather than if one should do so.

Jennifer Bussell’s (University of California, Berkeley) paper deals with the production of data based on direct observation and proposes “shadowing” as a particularly useful and transparent data collection technique for the study of political elites. The papers by Juan Masullo (University of Oxford) & Andrés Vargas (Yale University) and Ezequiel González-Ocantos (University of Oxford) & Juan Masullo focus on interview data. Masullo & Vargas’ paper focuses on reporting interview data. It identifies common critiques of lack of rigor and objectivity leveraged against interview-based research, and show a way forward in addressing them by proposing a set of reporting practices that provide targeted information for readers to assess the evidentiary power of interview data used to substantiate empirical claims. González Ocantos & Masullo focus on the collection/production of interview data for the specific task of process tracing. Building on insights from recent developments in the methodological literature on process tracing, the paper discusses concrete strategies to select respondents and design interview guides in both elite and non-elite interviews that could lead to the collection/production of data that is better suited to meet the high evidentiary demands of process tracing. The last paper, by Rachel Sweet (University of Notre Dame), deals with the evidentiary challenges related to attributing violence in conflict zones. Using a range of original data sources, including sensitive data from internal rebel records unearthed in the field, it develops a framework on how to use and evaluate the quality of data used to draw inferences on violent actors and events.

The panel will conclude with comments and feedback from two stellar Discussants who have been at the core of transparency and ethical debates in qualitative research and that have enormously contributed to the advancement of field research in political science: Diana Kapiszewski (Georgetown University) and Elisabeth Wood (Yale University).

Taken together, these papers represent a variety of core research subjects in political science and cut across disciplinary fields and sub-fields (from civil war to political corruption to political behavior). Each and all of them represent efforts to reflect on ways to render the production, analysis and reporting of data collected during field work more transparent, without breaching ethical commitments to human subjects, or undermining epistemological pluralism in the discipline.

Sub Unit

• DIVISION 46: Qualitative and Multi-Method Research

Individual Presentations

• Shadowing as a Tool for Studying Elites - Jennifer L. Bussell, University of California, Berkeley

• How to Report Interviews? Substantiating Claims with Interview Data - Andres Vargas, Yale University; Juan Masullo, University of Oxford

• Interviewing for Process Tracing - Juan Masullo, University of Oxford; Ezequiel Alejo Gonzalez Ocantos, University of Oxford

• Problems of Attributing Civil War Violence: Evaluating Data Quality - Rachel Sweet, University of Notre Dame

Chair

• Diana Kapiszewski Georgetown University

Discussant

• Elisabeth Jean Wood Yale University