Measuring Classroom Conversational Moves: Insights Into Treatment Effects, Student Outcomes, and Longitudinal Changes in Practice

• In Event: Toward a More Dialogic Pedagogy: Improving Student Learning in Science, Mathematics, and Language Arts Education

Abstract

As findings about the instructional value of “dialogic discourse” accumulate, researchers increasingly need to find ways to measure reliably the nature of classroom discourse. Many tools ask observers to use Likert-type scales to rate the degree to which classrooms display various global dimensions such as “engagement” and “authentic response.” As part of a large IES-funded study of reading comprehension, we have developed a tool that instead uses a relatively low-inference method of counting instances of well-known conversational ‘moves’ by students and teachers. For example, Teacher moves include those specifically thought to support student transactivity (Berkowitz & Gibbs, 1985). Transactivity involves a focus on reasoning, and entails that the speaker is focusing on the reasoning of others. The Teacher moves include prompt students to clarify their own contribution, prompt students to respond to others’ reasoning, and 6 others. Student moves include ask another student a question about their contribution, link a previous student’s contribution to the current point in the discussion, and 6 others.

We will present summative data from approximately 400, 4th-7th grade classroom lessons, recorded as part of the larger project, describing the distributions of target conversational moves which our theoretical frame would suggest support dialogic interaction. In the first part of the talk we will outline research using this type of low-inference measure that shows: (a) significant treatment effects on discourse practices across three study conditions; (b) preliminary data showing its use in predicting student outcomes on a standardized test; and (c) an analysis of a subsample of teachers observed over 2 school years, suggesting its possible use for tracking changes in teacher practice.

In the last section of the talk we will present several methodological challenges presented by using low-inference count data, along with some solutions. One such challenge concerns the skewed distribution of these theoretically important talk moves. For example, some Teacher moves, such as “closed question” or “display question” appear in the vast majority of lessons, while a “prompt students to respond to others’ reasoning” move is very rare. Because the distribution of these moves is strongly skewed, ordinary least-squares regression is not an option. Instead, we have uses a mixed analysis of rare-occurrence count data and non-rare count data using item response theory (IRT) (Muthén, 1989; Schrodt, 2007). For the initial analysis, rare categories were coded as dichotomous and non-rare categories were coded as trichotomous (not occurring, occurring once, or more than once). To account for the combination of different response scales in the transformed data, a single-parameter partial credit model (PCM) was used. IRT analysis based on this approach suggests that a single latent variable underlies all of the moves theorized to support dialogic, transactive discourse. Notably though, “brief response” and “closed question” student and teacher moves do not fall within this category. The implication is that such dialogue is dependent on skills and constructs which are distinct and separate from much pedagogic discourse.

Authors

• Catherine O'Connor, Boston University

• Maria D. LaRusso, Harvard University

• Allen G. Harbaugh, Boston University