Measuring Micro-Level Social Interactions: An Image-Driven Computational Approach

• In Event: Advances in Computational Methods (Co-Sponsored by Mathematical Sociology Section)

Abstract

Most computer vision classification tasks in the social sciences focus on \textit{standalone}, static objects; in contrast, we study \textit{interactive}, dynamic social processes—specifically, social interactions—embedded in still images. We develop a framework to detect and classify social interactions in multiple forms and apply it to a novel dataset of 7,500 photographs collected at 13 major academic conferences, capturing over 10,000 informal interaction groups. Methodologically, we introduce a dual-image architecture that combines full-scene context with masked pairwise crops to distinguish focal engagement from spatial co-presence, along with detailed interaction-form classification and distance-stratified negative sampling. Using Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning, we show that task-specific adaptation substantially improves precision and group-level interaction reconstruction relative to zero-shot performance. Substantively, we use reconstructed interaction networks to measure racial segregation in spontaneous professional encounters and find levels of segregation that exceed residential, educational, and subfield benchmarks and closely mirror segregation in coauthorship networks, suggesting that durable inequalities are already visible in short-lived, informal interactions. By shifting from static object recognition to relational interaction measurement, this study demonstrates how computational vision models can generate new measures of inequality at the level of everyday social encounters.

Authors

• Wenhao Jiang, Duke University

• Siwei Cheng, New York University

• Yue Yu, Hong Kong University of Science and Technology