Contemporary Student Perceptions of Teaching Behavior: A Topic Modeling Approach

• In Event: Effective Learning Environment: Contemporary Insights for Assessments and Strategies to Support Effective Teaching

Abstract

1. Objectives
This study aims to unravel contemporary teaching behaviour as a determinant of psychosocial learning environments, as perceived by students. A Machine Learning (ML) approach was applied to large-scale textual student responses. The validity of the ML analysis for mapping student perceptions of teachers’ teaching behaviour was explored.
2. Theoretical framework
Effective learning environments and more specifically, students’ perceptions of learning environments, affect learning outcomes (Entwistle, 1991). Latent Dirichlet Allocation (LDA) topic modeling analysis (Blei et al., 2003) was applied as a framework for machine-based analysis. A theory-driven manual qualitative analysis, based on the model of effective teaching behaviour of van de Grift (2007), was conducted to examine the validity of the LDA to unveil contemporary psychosocial classroom environments as perceived by students.

3. Methods
LDA was performed and the interpretation of the topics was assisted by an interactive visualization tool (i.e., LDAvis; Sievert & Shirley, 2014). Additionally, a theory-driven content analysis (Mayring, 2010) was conducted. This mixed-method approach was used to address the second aim of this study to shed light on the convergent and divergent validity of the ML analysis.
4. Data sources and measure
A total of 173,858 secondary education students were surveyed in The Netherlands. Of the student data, 31.7% were female, 28.5% were male; 31.1% were at 1st grade followed by 27.6% at 2nd grade; 32,4% were in lower secondary vocational education.
An open-ended question (“what the teacher does well is…”) was used to gather information about student perceptions of the psycho-social learning environment. Furthermore, 864 randomly selected student responses were subjected to manual coding.

5. Results
The LDA analysis revealed eight main topics, representing data-driven teaching behaviour domains. The relevance of eight topics is discussed in terms of the marginal distribution of topics, suggested topic labels, and topic representative student response examples (Table 1). Additionally, the theory-driven content analysis yielded nine teaching behaviour domains (Table 2).
The mixed-methods revealed overlapping and distinct frequency rates and the ML analysis revealed additional unique topics (Figure 1). Overlapping topics provide evidence for the convergent validity of the ML approach for analysing large-scale open-ended survey questions. Unique topics offer novel insights into contemporary psycho-social learning environments.

6. Significance
The study suggests that the machine-based approach is of value to teachers because LDA topic modelling analysis allows for extracting meanings from written text without the burden of reading every single big-data record. If the topics could be fed back to teachers, it may contribute towards learning environments that align more to students’ psycho-social preferences.
LDA topic modelling revealed unique topics that would not be reflected in a theory-driven analysis. The chosen theoretical framework might be narrower than contemporary student perceptions of good teaching. These LDA procedure provides an opportunity to track students’ perceptions over time considering the context- and time-dependent aspects of teaching behaviour. These findings have theoretical significance, showing a novel approach to detect contemporary teaching behaviours as an important determinant of learning environments.

Authors

• Bilge Gencoglu, University of Groningen

• Michelle Helms-Lorenz, University of Groningen

• Ridwan Maulana, University of Groningen

• Ellen Jansen

• Oguzhan Gencoglu, Top Data Science