An Application of Generalizability Theory to Analyzing Instructional Quality Observer Ratings for Mode Effect

• In Event: AERA Poster Session 2
  In Poster Session: Mathematics Learning

Abstract

Research on instructional quality has been conducted for many years. However, findings on observation modes are still rare. In the present study, we took this desideratum into account. Live and video ratings were carried out with an instrument consisting of 25 high-inferent items that cover three generic and two subject-specific dimensions of instructional quality. There were 15 teachers participating in our study of whom we observed thirty lessons during mathematics instruction. We carried out generalizability analyses for video and live scores. We found differences in reliability and mean scores. Because of small sample size, the findings must be interpreted cautiously, but they suggest that mode effects might be an issue when constructing a validity argument for an observational instrument.

Author

• Armin Jentsch, University of Cologne