The Role of Subject Matter in Measuring Teaching Across Countries

• In Event: Conceptualizing the TALIS Video Study

Abstract

Theoretical Framework and Objectives
Previous cross-cultural research has very much focused on opportunity to learn (OTL; Schmidt & Maier, 2009) to explain variation in student achievement within and between countries. The more (and deeper) content students are exposed to, and the more focused and coherent the curriculum is, the better are the results in international assessments (Schmidt et al., 2001). In contrast to other measures of teaching, OTL even explained achievement progress in a longitudinal cross-cultural design (Burstein, 1993). Starting with TIMSS-Video (Stigler & Hiebert, 1999), research increasingly studied how content was implemented, analyzing, e.g., the coherence of math lessons and the use of representations – issues taken up in national studies (e.g., Learning Mathematics for Teaching, 2011; Lipowsky et al., 2009). On an international scale, PISA 2012, based on its questionnaire framework (Klieme et al., 2013), was the first study covering OTL, teaching practices and teaching quality, enabling publications on OTL (Schmidt et al, 2015) and teaching (Caro et al., 2016) in relation to PISA test scores, with limitations due to the cross-sectional design. A national longitudinal extension (Kuger et al., 2017) confirms that student’s exposure to algebra and classroom discipline are related to mathematics achievement after controlling for the pretest, while – contrary to cross-sectional findings presented by Caro et al. (2016) - no such relationship was found for any math-specific measure of teaching such as cognitive activation through challenging problems.
We still need to learn how subject-specific teaching practices and teaching quality are related to student learning across countries. The TALIS Video Study is using a new approach to meet this challenge: Selecting a single topic to be taught in all 680 participating classrooms, with instruments tailored to the content.
Method
Inspecting the US Common Core standards for mathematics, content rubrics from international assessments, and previous comparative work on math curricula, 23 units from algebra, geometry, numbers, functions and probability were characterized by sets of learning goals. Mathematics education experts in participating countries were asked to identify whether, when and for how many lessons these units were taught according to national curricula, standards and/or textbooks. Nine topics were part of the obligatory curriculum close to age 15 and taught with a reasonable number of lessons in all countries. Two units were selected: pairs of simultaneous linear equations, and quadratic functions/equations. In a second stage of curriculum analysis, each country submitted details of its curriculum, which finally led to the choice of the latter.
Results and Discussion
Stage one of the curriculum mapping as well as secondary analyses of PISA 2012 OTL data lead to the conclusion that algebra is key to the lower secondary math curriculum. Coverage of topics in other areas changed a lot between countries, while algebra was taught for a substantive amount of time, at similar age/grade levels, with similar content elements across countries. Nevertheless, the second stage of the curriculum analysis still revealed severe differences in how countries implement these topics.

Authors

• Eckhard J. Klieme, German Institute for International Educational Research

• Kristina Maria Reiss, Technische Universitaet Munich

• Courtney A. Bell, ETS

• Anna-Katharina Praetorius, University of Zurich

• Brian Stecher, The RAND Corporation

• V. Darleen Opfer, RAND Education