Uncovering Career Interests From Open-Ended Responses: Integrating Topic Modeling and Large Language Models

• In Event: Artificial Intelligence Applications in Psychometrics: From Scale Development to Response Analysis

Abstract

This study applies the Structural Topic Model (STM) to uncover latent topics representing distinct career interests within students’ open-ended survey responses. The methodology facilitates not only the identification of these constructs but also the exploration of their relationships with various covariates. Additionally, we enhanced the analysis by integrating large language models to interpret the topics identified by STM, evaluating their effectiveness in generating precise topic labels. Our findings illustrate STM’s capability to provide a deep understanding of latent constructs expressed in textual data and demonstrate how the incorporation of LLMs can significantly improve the efficiency of STM analyses.

Authors

• Yuxiao Zhang, Purdue University

• Tugce Karatas, Purdue University

• Nielsen Pereira, Purdue University/Gifted Education Research & Resource Institute

• Sarah J. Bright, Purdue University

• Hua-Hua Chang, Purdue University