Design of the Teaching and Learning International Survey Video Study

• In Event: Conceptualizing the TALIS Video Study

Abstract

The first presentation will provide an introduction to the TALIS Video Study goals, place them in their socio-historical context, and give an overview of the design. The presentation will also discuss the ways that teaching has been conceptualized and measured in TALIS and PISA and how TALIS Video‘s teaching quality conceptualization and measurement is related to those longstanding international surveys.

The study design includes a number of unique elements that differentiate it from prior international video studies. The study focuses on the teaching and learning of a single common secondary mathematics topic -- quadratic equations -- across all countries and economies. We know from existing studies such as PISA that curriculum coverage has an impact on student achievement across countries, but the focus of the video study is different: this study is about how teachers teach and students learn rather than what is taught. The tight focus on a common topic supports the understanding of classroom teaching and learning practices across countries because it removes differences resulting from variations in the content matter taught.

The study is using a longitudinal design for student outcome measures, which will be captured before and after students’ learning of the focal content. The longitudinal design allows analyses to control for students’ prior knowledge. Similarly, non-cognitive dispositions (e.g., students’ self-efficacy beliefs) and motivation are assessed twice in the student questionnaires. The pretest scores and additional measures of students’ background will thus allow the study to separate out these sources of variance across classrooms from variance in teaching and learning to better isolate how teaching strategies and practices are associated with learning and other outcomes.

The study also uses standardized and replicated procedures for data collection, training and certifying video and artifact raters, and for coding videos and artifacts in every country and economy. This is important because in studies with less stringent processes, it can be challenging to determine whether differences across countries are real or simply the result of variation in implementation. Eighty five teachers from each country were randomly selected to participate. Two of the lessons they teach on quadratic equations will be video recorded and artifacts from those lessons and two others will be collected. Teachers are also asked to complete a pre- and post-questionnaire as well as a teaching log. Raters will be trained from each country to apply the observation and artifact codes.

Authors

• Darleen Opfer

• Eckhard J. Klieme, German Institute for International Educational Research

• Daniel F. McCaffrey, ETS

• Courtney A. Bell, ETS

• Brian Stecher, The RAND Corporation

• Anna-Katharina Praetorius, University of Zurich

• Thomas Van Essen, ETS