Generative AI for Researching Cognitive Presence in Online Learning Environments

• In Event: Technology Adoption and Online Learning

Abstract

The last two decades of online learning research vastly flourished by examining discussion board text data through content analysis based on constructs like cognitive presence (CP) with the Practical Inquiry Model (PIM). We leveraged large language models (LLMs) through GPT models to automate the content analysis of students’ text-based data based on PIM indicators and assess the reliability and efficiency of automated content analysis compared to human analysis. Using the seven steps of the Large Language Model Content Analysis (LACA) approach, we proposed an AI-adapted CP codebook in combination with a prompt using role and chain-of-thought techniques. We found that a fine-tuned model with a one-shot prompt achieved a moderate to substantial IRR with researchers. Implications for practice are discussed.

Authors

• Daniela Castellanos-Reyes, North Carolina State University

• Larisa A. Olesova, University of Florida

• Ayesha Sadaf, University of North Carolina - Charlotte