Detecting Diversity, Equity, and Inclusion in Course Syllabi: Applying Topic Modeling for Large-Scale Analysis

• In Event: Equity-Oriented Data Science: Leveraging Computation and Novel Data Sets for Higher Education Research and Practice

Abstract

Course syllabi are consistently understood and used across higher education. Often, syllabi are students' introduction to a course, instructor, and/or academic material. While these documents have traditionally been regarded as neutral or merely detailing course objectives and responsibilities, recent research has highlighted how syllabi can promote diversity, equity, and inclusion (DEI) within a given course (Fuentes et al., 2021; Veri et al., 2019). Applying natural language processing (NLP) methods, such as topic modeling, to a corpus of over 7 million syllabi across 3,000 US schools and over 60 subject areas, we explore the underlying DEI rhetoric structure or lack thereof.
Topic Modeling is a statistical method used to find underlying semantic structures in large collections of documents by constructing interpretable topics. A particular limitation of using topic models with collections that already have a structure is the inevitability that topic models will correlate with the underlying metadata such as course subjects. To overcome this limitation, we propose using topic modeling along with another popular NLP method, word embeddings, wherein word or phrase structure is further understood by the other words surrounding a word or phrase. Through applying topic modeling and word embeddings along with various preprocessing steps, we expect to find DEI structure in syllabi.
DEI efforts are useful because they allow and encourage the interrogation of policies and processes that have reinforced systemic oppression. While these efforts have become commonplace in higher education, many critics still believe that numbers or mathematical equations must be used to justify pursuing DEI goals in higher education. Though we, as authors, do not agree that DEI efforts must be mathematically justified, we do recognize the current data-driven environment of higher education. Informed by this context, this research contributes to DEI literature in higher education by offering an analysis of related discourse using NLP methods. In this study, we explore the algorithmic design of a tool that can be used as an aid to develop higher education syllabi that promote diversity, equity, and inclusion.

Author

• Kimberly Williamson, Cornell University