Considerations in Revising a Widely Used Standardized Observation System

• In Event: AERA Roundtable Session 5
  In Roundtable Session: Classroom Observation Manuals as Measures of Teaching Quality: Possibilities and Pitfalls

Abstract

Objectives:
The Classroom Assessment Scoring System (CLASS: Pianta, LaParo, Hamre, 2008) is one of the most widely used and researched measures of classroom quality. CLASS implementers are engaged in a revision of CLASS to: incorporate learnings from over two decades of research on CLASS, update with newer research on most salient elements of classroom interactions, and ensure CLASS is refined with current uses in mind – focusing on supporting use as a practice-based continuous quality improvement tool. In this presentation, we will discuss a possible framework for the revision and provide a summary of findings from initial stages of the work.

Perspectives:
The revised version of CLASS will continue to derive its core theoretical underpinnings from developmental science (Bell et al., 2018). Beyond updating the theory with more recent research in these areas the revision incorporates feedback from experts in equitable teaching practices (Curenton et al., 2020) and culturally responsive teaching (Jensen et al., 2020).

Data and Methods:
In this paper we will present findings from the first two stages of the revision process:
1) Field review and feedback – notably this included feedback from the research and practice communities aligned to their specific uses of CLASS.
2) Review of extant data – meta-analysis on CLASS is currently underway and will inform the revisions. In addition, we are in the process of analyzing the technical properties of the tool across a broad range of samples and contexts for use. This includes a rigorous assessment of our certification program that helps ensure consistent use of the tool across settings.

Findings:
Based on these reviews the revision focuses on the following goals:
1) Measure not only interactions but the classroom environment that support effective teaching and learning.
2) Measure more contextual information about student’s exposure to content of instruction (e.g. Math, Literacy, Science)
3) Refine indicators and markers for greater clarity and reduced redundancy
4) Train on observer bias with a focus on racial bias
5) Provide enhanced supports for scoring in classrooms with dual language learners and children with disabilities
6) More fully represent a range of cultures and contexts in the description of items and in training materials
7) Reduce complexity of CLASS age levels (move from 6 age levels to 3)
8) Allow for scoring at “item” level to increase precision of feedback
9) Ensure adequate distribution in data across score levels in typical samples
10) Ensure reports provide actionable data for educators.

Significance:
In this paper we will also call attention to the tensions in developing and refining observational systems in the context of broad use and discuss potential lessons for the field in how we develop, test, and refine the observational measures of the future.

Authors

• Bridget Kathleen Hamre, University of Virginia

• Robert Pianta, University of Virginia