Advances in Expert-Coded Data

Session Submission Type: Full Paper Panel

Session Description

In this panel, participants discuss recent advances in the social scientific use of expert-coded data. This topic is both timely and fitting for the conference theme: while an increasing number of important projects involve collecting and analyzing expert-coded data (e.g. Bright Line Watch, the Chapel Hill Expert Survey [CHES], the Electoral Integrity Project, the GLOBSEC Vulnerability Index, the Human Rights Measurement Initiative [HRMI], and the Varieties of Democracy [V-Dem] Project), the theoretical and methodological underpinnings of these endeavors can go unaddressed. Equally importantly, while an extensive amount of research has focused on developing methodologically sound and technically innovative approaches to aggregating expert codings, making the resulting data interpretable and usable to a variety of consumers has drawn relatively less academic focus. This panel engages with these concerns, drawing on the panelists' years of experience with expert coded data: both the chair and the panelists have been deeply involved in the V-Dem Project, while the discussant has been instrumental in both the CHES and HRMI projects.

In her presentation, Brigitte Seim both presents a typology for understanding the advantages and disadvantages of different types of human coders and empirically illustrates the implications of this typology; she also discusses the importance of understanding and accounting for variation in expert quality. Daniel Pemstein takes the panel discussion to the methodological cutting-edge, discussing how to incorporate insights about both expert cognition and institutional change into the measurement of trends in institutions across time. Kyle Marquardt's presentation focuses on the trade-offs involved in maintaining such methodological rigor while also making expert-coded data accessible to a wide audience. Finally, Marina Povitkina provides a capstone for the panel, illustrating how to use expert-coded data to analyze the relationship between natural disasters and democratization.

Sub Unit

• Division 8: Political Methodology

Individual Presentations

• Choosing Survey Respondents - Brigitte Seim, University of North Carolina-Chapel Hill

• Modeling Longitudinal Multi-Rater Data - Daniel Pemstein, North Dakota State University

• Visualizing and Using Expert-Coded Data - Kyle L. Marquardt, University of Bergen

• Natural Disasters and the Potential for Democratization - Oskar Rydén; Marina Povitkina, University of Oslo; Sverker C. Jagers, University of Gothenburg; Martin Sjöstedt, University of Gothenburg

Chair

• Jeffrey Staton Emory University

Discussants

• Ryan Bakker University of Georgia

• Zachary David Greene University of Strathclyde