Relating Instruction to Student Outcomes

• In Event: 66.051 - Tracing the Production Function: Connecting Teacher Preparation, Knowledge, Beliefs, and Instruction to Student Achievement

Abstract

Forty years of research on teaching has not resulted in a consensus about the elements of instruction most important to producing student outcomes. Scholars first studied generic pedagogical interactions such as question wait time, structuring, and summarization (see Brophy & Good, 1986, for a summary), then moved to disciplinary-focused pedagogical dimensions, such as the cognitive demand of student tasks (Stein & Lane, 1996; Berliner, 1981), and then onto classroom climate, productivity, and affective variables (Pianta, Belsky, Vandergrift, Houts, & Morrison, 2008; Ames, 1992). Studies within each sub-field have returned positive results vis-à-vis student outcomes, but few comparisons across different subfields exist, leading to disagreements among scholars of teaching over what to prioritize within instruction. These disagreements are still apparent in the practical advice given to teachers: Teach like a Champion. Teach for Meaning. Even Teach like Your Hair’s on Fire. Teacher observation instruments reflect these divisions as well, ranging from those that espouse lessons from the process-product era (e.g., Marzano) to those that incorporate the idea that students should be engaged in cognitively demanding tasks (e.g., Framework for Teaching).

In this paper, we use observational and student outcome data from roughly 300 teachers participating in the overarching three-year main study to assess the extent to which different aspects of instruction relate to student outcomes. For teachers in our sample, we measured the aspects of classroom instruction identified as important in the literature using up to six recorded lessons per teacher, further broken down into shorter segments. We measured classroom climate and productivity using the Classroom Assessment Scoring System (CLASS, Pianta et al., 2008) as well as general pedagogical interactions and discipline-specific elements of instruction using the Mathematical Quality of Instruction (MQI, Hill et al., 2008).

To better understand the components of instruction captured by the two instruments, we began analyses by conducting exploratory and confirmatory factor analyses of the primary MQI and CLASS items using scores from roughly 1300 lessons. We found that a four-factor structure capturing measures of teachers’ ambitious mathematics instruction, mathematical errors, classroom organization, and classroom climate best fit the data.

We then investigated the relationship of these four factors, in addition to more generic pedagogical practices such as checking for understanding and orienting students to new content, to student achievement on two assessments: state standardized math assessments, and a study-administered low-stakes mathematics assessment. Using hierarchical linear modeling, we found that of the different instructional factors described above, teacher classroom organization scores most consistently related to improved student outcomes on both tests.

In coming months, we plan to model interactions between our four measures of instruction, hoping to represent theories that view instruction as the complex interaction of multiple dimensions (e.g., Cohen, 2011). We will also conduct exploratory analyses that examine whether the production function that links instruction to student outcomes is sensitive to test type and district context.

Authors

• Heather C. Hill, Harvard Graduate School of Education

• Mark Chin, Harvard University

• Erica Litke, University of Delaware

• Kathleen Lynch, Harvard University