Educational Video Transcript Analysis with LLMs: Improving Entity Recognition and Qualitative Insights

• In Event: AERA Roundtable Session Friday 9:45 am - Gold 4
  In Roundtable Session: Mixed Methods Frontiers: Leveraging AI in Multiple and Mixed Methods Research

Abstract

This study presents an integrated pipeline for large-scale qualitative analysis of educational video content, leveraging Automatic Speech Recognition (ASR) and Large Language Models (LLMs) for coreference resolution and Named Entity Recognition (NER) correction. Using 48 CrashCourse US History episodes, we benchmarked multiple ASR systems and applied LLM-based enhancements. Four episodes were manually annotated as gold standards to validate improvements. Results demonstrate that LLM-assisted coreference and NER significantly boost the accuracy and reliability of historical entity extraction, particularly for complex events, organizations, and laws. Topic modeling reveals that LLM-cleaned transcripts yield clearer, more coherent themes. Our findings highlight LLMs’ value in enhancing transcript fidelity, entity recognition, and thematic analysis, supporting more rigorous educational research across diverse contexts.

Authors

• Wei Wang, University of Tennessee

• Cody C. Pritchard, University of Tennessee

• Joshua Rosenberg, University of Tennessee