Pluralistic Approaches to Complex Inferential Challenges

Session Submission Type: In-Person Roundtable

Session Description

A discussion of pluralistic methodological innovations.

"A Matter of Perspective: Computational Social Science and Researcher Choice"

We are living in a golden age for social science. A time when researchers have both unprecedented access to human data and the computational power to interrogate those data. Innovation has proceeded with remarkable speed; new methods regularly deliver novel insights with increasingly improved accuracy. These data and tools promise to reinvigorate the social sciences, allowing researchers to ask new questions in meaningful, new ways and to revisit classic questions with renewed rigor. While computational capacity is no longer the insurmountable barrier it used to be, the opportunity presented by this methodological innovation underscores the urgent need to address another long-standing limitation of scientific advancement: the relative homogeneity of the research community. Methodological pluralism–diversity of both methods and methodologists–is essential for ensuring the collective creativity necessary to fully leverage the tools and data available to researchers. How we ask and answer questions matters. How we conceive of questions, what data and tools we bring to those questions, and how we interpret our findings are all intimately interconnected with researcher perspectives and identities. If social science ultimately aims to understand the human experience, it cannot do so without full consideration for the diversity of that experience. With particular focus on text and network methods, we illustrate the role of researcher choice in shaping scientific advancement and demonstrate the need for human interpretation of algorithmic output. By interrogating the role of human researchers in scientific discovery we showcase the value of methodological pluralism and argue that diversity is an essential and necessary condition for scientific advancement.

"What type of data are images?" (Pre-Recorded)

Images are an intriguing source of data for social scientists. Increasingly, the phrase “images as data” implies big data, digitized images, and quantitative analyses with elements of machine learning. This perspective, however, obscures the many other ways to conceive of what images are as data for social scientists. For example, big data, machine learning image researchers rely on labeled data to build classification algorithms. The process of labeling the data is essentially content analysis, a familiar tool to qualitative researchers. Thus, quantitative analyses are built on qualitative approaches and both are a potentially valid way to use images to better understand the world.

"Causal Inference with Panel Data"

Panel data are commonly used in the social sciences to study the causal effects of policy interventions on certain outcomes. Yet the assumptions under which popular panel data methods—such as difference-in-differences and two-way fixed effect models—can perform properly are often unsatisfied. We discuss the emerging literature on causal inference with panel data and focus on a special setting of a single, dichotomous treatment variable. We survey three broadly defined approaches: (1) non-parametric methods such as matching and reweighting; (2) parametric or semi-parametric methods that focus on modeling the response surface; and (3) hybrid methods.

"Human Rights in Space: Statistical Models of Machine Coded Vs. Human Coded Data”

Political event data are widely used in studies of inter- and intra-state political processes such as human rights abuses. Recent years have seen notable advances in the automated coding of political event data from international news sources. Yet, the validity of machine coded event data remains disputed, especially in the context of event geolocation. We estimate neighborhood- and continuous-spatial models using both machine and human coded event data. The former are spatial probit error models, estimated via a likelihood-based approximation and a univariate conditioning procedure. The latter models are also spatial probits but their error processes are assumed to be Gaussian Markov Random fields. These continuous-spatial models are constructed using the stochastic differential equation methods and estimated with integrated nested Laplacian approximation. We combine these models with data on subnational violence in Colombia to evaluate whether human and machine coded event data produce comparable inferences, predictions, and patterns of prediction error. All spatial predictions are validated against independently collected, gold standard records of human rights violations from Colombia. We find that machine geocodings are as accurate as human geocodings for analyses that locate events in a discrete set of subnational spatial units but are inferior to human geocoding in analyses that locate subnational events in continuous space.

"Pluralism in Time Series Analysis"

Sub Unit

• Theme Panel

Chair

• Matthew P. Hitt Colorado State University

Presenters

• Nora Webb Williams University of Illinois at Urbana-Champaign

• Sarah Shugars New York University

• John R. Freeman University of Minnesota, Twin Cities

• Yiqing Xu Stanford University