Assessment and validity of advanced large language models for deductive narrative content analysis: A comparative study of ChatGPT and Claude.ai to human coding

• In Event: Advances in the Narrative Policy Framework - New Applications of the NPF

Brief Overview

We aim to evaluate the proficiency of LLMs to follow instructions from codebooks developed and tested for NPF coding for human coders. Using the same data set, we compare human coding to AI coding.

Authors

• Liz A Shanahan, Montana State University

• Manuel Ruiz-Aravena

• Rob A. DeLeo, Bentley University

• Elizabeth Ann Albright, Duke University

• Kristin Taylor, Wayne State University

• Nathan Jeschke, University of Colorado Denver