Generative Large Language Models and Political Science Education

• In Event: TLC at APSA: Assessing and Implementing Information Literacy, Portfolios, and Research Labs

Abstract

Recent applications of new innovations in artificial intelligence (AI) can generate original content in response to custom prompts at user-specified lengths in seconds. These tools, capable of writing human-like prose and answering a wide variety of questions, are commonly referred to as chatbots or generative large language models. Popular platforms like ChatGPT, Jasper, and Moonbeam are free or low-cost, sparking concerns among many in higher education. Using original data from two surveys (in collaboration with the American Political Science Association (APSA)) and analyses of student- and bot-written essays, this project aims to examine the likely impact of these new chatbots on political science education. Specifically, it presents data about educators’ views on the pedagogical value of written assessment, positive and negative applications of AI (e.g., copy-editing versus completing essays from scratch), and the perceived impact of these AI tools on political science courses. Lastly, we highlight the strengths and weaknesses of AI-generated essays based on respondents’ performance in an exercise to identify essays written by students or chatbots and analyses of ChatGPT output.

Authors

• Nicole Wu, University of Toronto

• Patrick Y. Wu, American University