Applications of Critical Classroom Video Analysis and Natural Language Processing to Observe Student Engagement in Classroom Discourse

• In Event: Real insights from real classrooms: Motivation scholars’ experiences designing for and working with observational data

Abstract

Science discourse is at the heart of students’ science learning; as thought turns into talk it not only finds its expression but its reality and form (Lemke, 2004; Vygotsky, 1986). By asking questions, sharing connections to lived experiences, and generating evidence-based explanations, students take ownership of making sense of the natural world (Bae et al., 2021; Windschilt, 2020). This presentation draws from our multi-year (2019-25) research practice partnership with middle school teachers focused on developing discourse-rich science activities that are meaningfully connected to students’ lives. We collected classroom videos from a school district in the southeastern region of the United States, representing a racially and socioeconomically diverse student population (56% Black, 27.4% Latinx, 11.5% white, 3% Multiple races, 1% Asian, and .3% Native American; 21.8% English Language Learners; 65.6% Participating Students in Free or Reduced Lunch Program).
The first study illustrates the application of a hybrid discourse framework (Moje, 2004) to guide the design of a classroom observation study that attends to both students’ moment-to-moment engagement in science talk and the socially situated language practices that reflect identities, ideologies, and cultures (Gee, 2004). The purpose of this study was to extend scholarship on equitable classroom talk (e.g., Jensen et al., 2021) by offering a more granular lens for identifying how academic and everyday discourses intersect in hybrid spaces. Key methodological decisions related to video data collection (Swivl protocol), transcription preparation (Whisper AI software and human cleaning, Radford et al., 2023) and determining time-defined and qualitative units of analysis (e.g., Brown & Spang, 2008; Sun et al., 2023) to systematically identify hybrid discourses will be shared. We will also share the research team’s positionality (Boveda & Annamma, 2023) and discuss how we engaged in reflexive discussions and triangulated interpretations of the videos with classroom teachers to draw valid conclusions from the video analyses.
The second mixed methods study uses the same hybrid discourse framework to guide the interpretation of natural language processing (NLP) results (e.g., word frequency count, parts-of-speech tagging, topic modeling, language style matching index, Liu & Cohen, 2021; Pennebaker et al., 2015). In this study, we identified the discursive resources or funds of knowledge (Moll et al., 1922) that today’s youth bring to classroom discussions (e.g., analytic versus social language), as well as to trace student agency in classroom talk (e.g., open-ended questioning, teacher-student language coordination). The affordances and limitations of descriptive NLP results will be discussed, coupled with examples of how qualitative analyses are used to contextualize NLP results.
In summary, we discuss how observational methods must not only document what is observed but also interrogate how systems and sociocultural forces shape what becomes visible in classrooms. We conclude with a set of methodological insights for motivation researchers seeking to leverage observational data, including the affordances of combining NLP and video analysis, techniques for capturing discourse-related indicators of student engagement, and the importance of framing observations within classroom contexts.

Authors

• Christine Lee Bae, Virginia Commonwealth University

• Kamil Hankour, Virginia Commonwealth University

• Kimberly Williamson, Lehigh University

• Morgan Les DeBusk-Lane, Gallup

• Rachel Niemira, Virginia Commonwealth University

• Ryan LeVault, Virginia Commonwealth University

• Singith Nuwanga Perera, Virginia Commonwealth University

• Gabriella Osei-Poku, Virginia Commonwealth University