Developing a Teacher Log: Validation Efforts and Results

• In Event: Middle School Mathematics Teachers' Common Core State Standards for Mathematics Interpretations and Enactments: Complementary Findings From Multiple Instruments

Abstract

Objectives: We describe the development, use, and validation of a teacher log tool, a survey tool administered before and after teachers taught lessons intended to be aligned with the CCSSM.

Theoretical Position: We developed and administered the teacher logs with three limitations in mind. First, teacher logs or surveys cannot provide the same depth as observations or interviews (Desimone & LeFloch, 2004). Second, questions persist as to whether teachers’ reported practices reliably describe actual practices (Desimone, 2009). Third, survey analysis allows researchers to distinguish between teacher practices, though the quality of engagement in those practices cannot be reliably estimated (Mayer, 1999). To address these limitations, we included a number of open-response items, compared the teacher logs to interviews based on the same lessons, and cautiously assessed teachers’ practices.

Methods of Inquiry: We asked teachers about lesson planning – including standards, anticipated student strategies, and formative assessment questions – and lesson enactments, including student strategies, surprising events, and possible revisions.

We conducted two analyses. In the first, we compared the information from the log with the information from the interviews. In the second, we analyzed the extent to which teachers provided details of the mathematics and of student thinking.

Data Sources: There were between 7 and 14 open-response items in three versions of the logs, in addition to closed-response items that identified standards and so forth. We analyzed a total of 206 logs. For a subset of the logs, we conducted pre- and post-lesson interviews for validation purposes.

Results: Our first analysis compared interviews to the teacher logs. We found that the interviews elicited more specific information, especially in the follow-up questions, and that the information found in one interview response was often distributed across multiple teacher log responses. We also found that teachers did not elaborate on their assessment practices in the teacher logs. In our second analysis, we analyzed the extent to which the teachers described the mathematics and their students’ engagement with the mathematics. In particular, we analyzed the extent to which they elaborated on the mathematics or the student thinking related to the mathematics. We used these results to consider how teachers were interpreting the mathematical content standards and how teachers reflected on the strategies and challenges faced by their students. First, we found that teachers who provided details and elaboration about the mathematics of the lessons did the same for student thinking (0.72 correlation). Second, we found that the majority of teachers provided few details of the mathematics or student thinking beyond stating the standards or mathematical content of the lessons. Third, the teachers’ provided little interpretation of the CCSSM or how they viewed their instruction as aligned with the CCSSM, with most simply listing the standard when asked how the lesson aligned with the CCSSM.

Significance of the Work. The teacher logs may provide a relatively inexpensive way to gauge teachers’ interpretations of the CCSSM and their curricular and instructional practices they deem aligned with the CCSSM. These analyses highlight the promises and limitations of the instrument.

Author

• Jeffrey M. Choppin, University of Rochester