Teaching practices from a cross-cultural perspective: methodological rigors and innovations

Group Submission Type: Formal Panel Session

Proposal

A high-quality teaching force is vital for the quality of every educational system, as teachers, teacher’s collaboration, and teaching practices can exert great influence on student learning outcomes. From a cross-cultural perspective, a major interest of educational effectiveness research is explaining why and to what extent teaching beliefs, practices, and qualities differ across cultures and how they are associated with student outcomes.

Large-scale educational assessments mostly assess teaching-related variables through contextual questionnaires. However, one main challenge in comparative education is assuring data comparability across countries. In cross-cultural surveys, language differences, group-membership, contextual factors, and other sources of non-observed heterogeneity of respondent groups, can shape the understanding and response process of teaching-related measures. Ideally, variation between cultural groups should lie on the construct of interest, rather than on interactions between respondents and instrument attributes. Yet, if questionnaire data are not equivalently interpretable across countries, then comparison is futile. Hence, before valid cross-cultural comparative inferences can be drawn, it is necessary to formally demonstrate comparability of target measures.

Educational large-scale assessments devote many of its efforts to assure that test scores, observations and other forms of student outcome measures are validly comparable between countries and regions. However, despite the critical relevance, there is much less effort in assuring comparability of background variables influencing student learning, such as teaching practices or teacher’s collaboration. Yet, to draw valid comparative inferences across countries, it is imperative to demonstrate comparability of background measures.

This symposium gathers four examples of current research on methodological issues on teaching practices assessment. We showcase applications of different quantitative methods to assess cross-cultural comparability of teaching-related measures (e.g., teacher’s collaboration or teaching practices). To uncover non-obvious response bias and address limitations of quantitative methods, we complement our quantitative comparisons by mixed-method designs involving qualitative in-depth interviews.

We point out the significance of enhancing the methodological rigor and extending our toolbox to better understanding teaching practices across cultures by
1) checking if questionnaire items measuring constructs on teachers and teaching are comparable across cultural groups by using different quantitative analysis methods,
2) identifying factors influencing data comparability (e.g., geographical vicinity, language),
3) investigating alternative psychometric methods (e.g., network analysis) to analyze cross-cultural data of teaching practices,
4) complementing quantitative evidence with qualitative interviews to broaden our understanding of teaching across cultures.

All in all, large-scale educational studies increasingly provide opportunities for comparing student outcomes across heterogeneous populations (e.g., involving multiple school tracks and countries). Questionnaire data are used as predictors and control variables when explaining student achievement, or reported as non-cognitive learning outcomes. Group-membership and contextual factors, however, can influence the way survey items are interpreted, considered, and ultimately answered across cultures (Miller et al., 2011) and thus can seriously jeopardize data comparability.

Hence, in order to validly explain cross-group differences, to learn from different educational systems, and to provide policy relevant information, it is imperative to (1) demonstrate comparability of target measures across groups, and (2) utilize diverse methodologies to uncover the complexity of teachers and teaching.

Sub Unit

• SIG: Large-Scale Cross-National Studies in Education

Chair

• Jia He, German Institute for International Educational Research

Individual Presentations

• Measurement invariance of a teacher professional learning community scale in large-scale studies - Enrique Eduardo Valencia López, University of California, Berkeley; Catalina Lomos, LISER Luxembourg

• Comparing latent means across countries - How different are the results when comparing groups of countries clustered by geographical vicinity? - Catalina Lomos, LISER Luxembourg

• Comparing teaching practice items across countries: Using network analysis - Jessica Fischer, German Institute for International Educational Research; Jia He, German Institute for International Educational Research; Eckhard Klieme, German Institute for International Educational Research

• From large-scale statistics to small-scale interviews: Investigating the influence of sociocultural context on teachers’ responses to accountability instruments in two countries - Yue-Yi Hwa, University of Cambridge, United Kingdom

Discussant

• Diego Carrasco, Centro de Medición MIDE UC, Pontificia Universidad Católica de Chile