Kindergarten Teachers' Effectiveness Ratings During Science Instruction: A Generalizability Study of Stability Over Time

• In Event: 31.091 - Roundtable Session 10
  In Roundtable Session: 31.091-9 - Accountability: Issues of Measurement and Model Quality

Abstract

To comply with national accountability mandates, states require 2-3 yearly observations of a teacher’s effectiveness (TE). However, there is virtually no evidence that this number produces reliable TE estimates. Stability data are critically needed, particularly because TE scores guide high stakes decisions. In response, we examined 40 video-recorded science lessons (taught by 10 kindergarten teachers, 4 times during the year) that we coded with the Classroom Assessment Scoring System. We used Generalizability Theory to document the variance attributed to teachers, time, raters, and TE domains. Preliminary evidence suggests that 2-3 lessons are insufficient for stable estimates. At least 5 lessons would be required to measure Emotional Support and 6 lessons to measure Instructional support. Classroom Organization was highly variable.

Authors

• Brian F. French, Washington State University

• Panayota Y. Mantzicopoulos, Purdue University

• Helen Patrick, Purdue University

• Inok Ahn, Purdue University

• Sam Watson, Purdue University