Alignment of Student and Teacher Perceptions on Different Domains of Teaching Quality in Cross-Country Comparison

• In Event: Measuring Teaching Practices and Teaching Quality: Conceptualizations and Findings From an International Study

Abstract

Objectives: We examine the relationship between teacher- and student-reported measures of multiple dimensions of teaching quality, separately across seven countries/economies that participated in the TALIS Video Study (TVS). The study examines whether teacher and student perspectives overlap on several domains including classroom management, social-emotional support, student participation in discourse, and quality of teacher explanation for mathematical procedures, with measures taken both before and after teaching a unit on quadratic equations. Due to the study design, not only differences by the domain of teaching quality, but also the timing of the measures as well as the cultural context can be investigated.
Theoretical framework: Teaching quality is usually investigated using either observer, student, or teacher ratings. The agreement among these ratings has scarcely been investigated systematically so far, leaving open under which conditions these measures come to similar conclusions on teaching. Existing studies indicate differences in agreement across teaching domains (for an overview see Fauth et al., 2020). Kunter and Baumert (2006), for example, found high correlations (.64 on class level) between teacher and student perceptions of classroom management in secondary mathematics, low but significant correlations (.25) for support, and zero correlations for cognitive activation. The generalizability of such findings across studies, timing within a study, and cultural contexts has so far not been investigated systematically.
Methods: Student and teacher responses to survey items are aggregated into average scale scores and student-level scores are then aggregated to the classroom level. We examine how student and teacher scores on the same scales are related, estimating both raw and regression-adjusted associations (controlling, e.g., for student and teacher background characteristics) between both measures. We compare how the strength of these associations differs by domains of teaching quality, timing of the measures, and country/economy.
Data sources: We use data from both waves of the TVS student and teacher surveys administered before/after the focal unit in seven countries/economies to construct scale scores. Student surveys asked students to report the teaching quality of their current teacher whereas teachers were asked to self-report their quality on the same items.
Results: Agreement differs considerably depending on the domain studied: agreement is high for Classroom Management and Students’ Opportunities to Participate in Discourse, whereas correlations are low for Quality of Teacher Explanations and for Social-Emotional Support. Additionally, the size of correlations differs before and after the focal unit across all countries. Finally, relations vary across cultural contexts, with the highest correlations in Germany and the lowest in in Colombia.
Scholarly significance of the study: The TVS provides a unique opportunity to examine whether teachers and student perspectives are in agreement across a variety of settings (i.e., domains, timing, and cultural contexts). The diversity in findings emphasizes first the complex nature of the relations between student and teacher perceptions on teaching. It second indicates that further conceptual work is needed on explaining such differences and eventually finding ways on how to optimally combine different perspectives.

Authors

• Kata Mihaly, RAND Corporation

• Sy Doan, RAND Corporation

• Eckhard J. Klieme, DIPF|Leibniz Institute for Research and Information in Education

• Anna-Katharina Praetorius, University of Zurich