Profiles of Practice: Exploring Innovative Evaluation Methods for Improving Mathematics Teaching and Learning

• In Event: Measuring Teaching Practices and Teaching Quality: Conceptualizations and Findings From an International Study

Abstract

Objectives: We examine mathematics teaching by focusing on patterns of practice (Douglas, 2009), as opposed to typical practice. Specifically, we 1) model patterns of practice across time to gain insight into how instruction is sequenced and 3) model patterns of practice across instruments to gain insight into how practices co-occur within teachers. We describe these patterns of practice as teaching profiles, and we investigate the extent to which these profiles are associated with student outcomes.
Theoretical framework: Mathematics teaching is complex (Leinhardt & Greeno, 1986; Shulman, 1987) and multiple measures are believed necessary to characterize it completely (Goe & Croft, 2009). However, most research takes a variable-centered perspective (Holzberger et al, 2019) to summarize data from multiple measures and to associate teaching with student outcomes. While such a perspective accurately characterizes typical practices, the variable-centered perspective is misaligned with the notion that teaching requires the coordination of multiple practices (Darling-Hammond & Bransford, 2007), and that teachers have different strengths and weaknesses (Grossman et al., 2013). A small number of studies used approaches that model patterns of practice (e.g., Halpin & Kieffer, 2015; LoCasale-Crouch et al., 2007). Those that focus on mathematics often rely on a single measurement mode (e.g., Holzberger et al., 2019). Our study is among the first to explore patterns of mathematics teaching using multiple measures.
Methods: We use latent profile analysis to characterize patterns of practice as teaching profiles following a three-step estimation approach: 1) specify an initial number of profiles and estimate models; 2) compare choices for the number of profiles using diagnostic measures (e.g., Lubke & Neale, 2006); and 3) interpret the latent profiles based on theory and prior research. We explore the extent to which teaching profiles are associated with student outcomes using latent profile regression analysis (e.g., Clark & Muthén, 2009). We conduct all analyses separately within each country/economy.
Data sources: We use data from video observations and artefacts collected during the focal unit of the TVS. Two lessons focused on quadratic equations were video-recorded and coded by raters. Artifacts (including lesson plans and teaching materials) were collected from each lesson that was video-recorded and the lesson that followed it (four lessons in total). Artifacts were also coded by raters.
Results: Regarding patterns of practice across time, LPA analysis identified evidence pointing to differences in instructional sequencing. For example, there are different profiles describing how teachers introduce opportunities for students to make connections outside of mathematics (i.e., real-world connections) across instructional days. Some teachers introduce real-world applications earlier in the unit than others. Regarding our cross-instrument investigations, we found that the nature and number of teaching profiles varies significantly across cultural contexts. Regression analyses find differential associations of teaching profiles with achievement gains.
Scholarly significance of the study: This project contributes to our understanding of how to characterize teaching quality. This study has the potential to advance our understanding of the ways in which teaching is organized inside classrooms to impact student learning.

Authors

• Jonathan Schweig, The RAND Corporation

• Courtney A. Bell, University of Wisconsin/ WCER

• Kata Mihaly, RAND Corporation

• Sy Doan, RAND Corporation

• Katherine Furgol Castellano, Educational Testing Service