(iPoster) Artificial Polarization: Large Language Models Systematically Suppress Moderate Opinions

• In Event: Poster Sessions: Political Methodology & Foreign Policy I – Great Power Competition and Strategic Rivalry
  In Poster Session: Poster Session: Political Methodology

Abstract

LLMs are now deployed at scale to model human opinion, yet we show they systematically suppress moderate responses and overproduce polarized ones. In surveys such as the American National Election Studies, up to 38.1% of individuals select neutral midpoints such as "neither agree nor disagree," while LLMs assign near-zero probability on most items when given detailed demographic profiles of individuals. Standard accuracy metrics cannot detect this artificial polarization: collapsing ordinal scales to binary agree/disagree inflates accuracy by up to 57.1 percentage points, but a Bayesian benchmarking recovers the full response distribution. These findings show that accuracy alone is insufficient for evaluating models of human opinion because it can conceal substantial failures in distributional fidelity. Across approximately 300,000 predictions from four models spanning 15 countries (2021--2024), we show that artificial polarization persists across models, surveys, and cultures and is not resolved by higher accuracy, raising concerns about AI use for modeling human opinion.

Author

• Ali Amini, Center for Data Science, American University