Large Language Models and the Dynamics of Affective Connotations in the United States, Germany, and France

• In Event: 40807 - Advancing Mathematical and Computational Research in Social Psychology (Co-sponsored by Section on Social Psychology)

Abstract

The affective connotations of words are central to their meanings. They inform how people interpret and use them. As such, they are of substantial interest to social scientists, who typically measured them using surveys. It is possible that large language models (LLMs) may offer a way to replicate and potentially improve upon survey-based measurements at substantially lower cost. Here, we test the extent to which LLMs can replicate human judgements of affective connotations of social actors, behaviors, and settings in three sociolinguistic contexts: English (US), French, and German. We also test the extent to which LLMs can accurately determine how affective connotations change in the context of social interaction. We find that LLMs reflect human affective connotation ratings reasonably accurately, though they are somewhat worse in the context of events. However, in both cases their ratings are overly extreme. Our results inform understandings of the strengths and weaknesses of LLMs as tools for social research.

Authors

• Aidan Combs, The Ohio State University

• Christophe Blaison

• Diego Dametto

• Renee Leung

• Aarti Malhotra

• Tobias Schroeder, Potsdam University of Applied Sciences

• Jesse Hoey

• Lynn Smith-Lovin, Duke University