Navigating Pathways for Automated Inductive Coding With Generative AI/Topic Modeling: An Exploratory Study (Poster 8)

• In Event: AERA Poster Session Friday 1:30 p.m.
  In Poster Session: Advanced Technologies for Learning Poster Session 2

Abstract

Inductive qualitative methods have been a mainstay of education research for decades, yet it takes much time and effort to conduct rigorously. Recent advances in artificial intelligence, particularly with generative AI (GAI), have led to initial success with inductive coding tasks. Like human coders, GAI tools rely on instructions to work, and how to instruct it may matter. To understand its impacts, we applied two known machine learning / GAI approaches and two theory-informed novel approaches to an online dataset and explored the resulting codebooks through four metrics. We show the strengths and weaknesses of current ML/GAI approaches for qualitative research and suggest the necessity of our ongoing work, a computational(-assisted) approach for evaluating open-ended qualitative codebooks.

Authors

• John Chen, Northwestern University

• Alexandros Lostos, Northwestern University

• Lexie Zhao, Northwestern University

• Grace Wang, Northwestern University

• Uri J. Wilensky, Northwestern University

• Bruce Sherin, Northwestern University

• Michael S. Horn, Northwestern University