The Policy Agenda of U.S. Presidential Speeches: A Deep Learning Approach

• In Event: Applications in Political Methodology Using Large Language Models

Abstract

Existing studies of presidential speeches rely on detailed sentence-level human-coding of a few highly visible speeches or apply a more systematic coding using various dictionaries for broad, document-level categorization. Neither approach provides a sufficient understanding of the scale—presidents speak, on average, every day—and scope—presidents refer to multiple policies in every speech—of the policy agenda of presidential speeches. We present a third approach that relies on fine-tuned large language models to code for policy thousands of occasional remarks of presidents Truman to Trump at the sentence level. We show that large language models offer superior precision and recall scores to code presidential speeches, which can provide a comprehensive assessment of the rhetorical policy agenda of presidents.

Authors

• Miklós Sebők, Centre for Social Sciences, Budapest

• Akos Mate, Centre for Social Sciences, Hungarian Academy of Sciences Centre of Excellence

• Amnon Cavari, Reichman University