Identifying Sources of Variability in Classroom Observational Data: A Grounded Theory

• In Event: 42.062 - Classroom Observations: Reliability, Validity, and Use in Determining Teacher Effectiveness

Abstract

Interrater reliability is notoriously difficult to establish for observational protocols; a multitude of factors influence the ratings that evaluators assign when assessing observable teaching practices. The purpose of the current study is to develop a grounded theory to describe the causal attributions evaluators make for variability in ratings on teacher observational protocols. A primary goal of this study is to develop an initial, testable theory with implications for increasing inter-rater reliability in observational data and supporting fairness and equity in teacher evaluation. Results indicate four primary categories of evaluator attributions for variability on observational ratings: (1) Evaluator interpretation of instrument scoring procedures (2) Evaluator interpretation of instrument language and content (3) Evaluator biases and (4) Evaluator prior experiences.

Authors

• Julie Brockman Smart, Clemson University

• Jeff C. Marshall, Clemson University