CLASS 2nd Edition: Reliability and Comparability to CLASS 2008 Moving Towards Equity within Classroom Interactions

• In Event: Enhancing Equity and Improvement: A Deep Dive Into CLASS (Classroom Assessment Scoring System) 2nd Edition Reliability and Validity

Abstract

The Classroom Assessment Scoring System (CLASS) is one of the most used measures of classroom interactions to support early learning educators. Teachstone recently updated the measure, CLASS 2nd Edition, with a focus on equity - ensuring every person (including children and educators) has the access to what they need to succeed and thrive - and inclusive language. However, the updated version maintained the CLASS framework. The goal of the current study was to examine the reliability of CLASS 2nd Edition scores and concurrent validity with CLASS 2008 scores.

The current study used video observations from 380 classrooms across 27 Head Start programs. Programs were selected from a larger sample participating in annual CLASS observations during the 2022-2023 program year. Programs submitted 60-minute videos for their identified classroom sample (sample size ranged from 4 to 27). Submitted videos were reviewed for quality and two cycles for CLASS coding were identified. Videos were then randomly assigned for coding. Each video was coded on CLASS 2008 and CLASS 2nd Edition.

Teachstone established a pool of certified CLASS observers with at least one year of CLASS observation experience and an initial pass rate of 80% reliability or higher on a calibration check prior to coding. Twenty-one observers were certified 2nd Edition observers with an average reliability of 89.9%. Observers completed bi-weekly calibration checks with an average level of reliability at 91.5%. To ensure the independence of scores observers completed a conflict-of-interest form for each video to verify that they had not scored and/or watched the video previously. Videos were automatically reassigned if a conflict was indicated. No videos were scored twice by the same observer and the CLASS 2008 and 2nd Edition scores were independent.

Analysis of internal consistency demonstrates that CLASS 2nd Edition has a high level of reliability (See Table 1). While correlations between CLASS domain scores on both versions ranged from .41 to .46 were significant and moderate demonstrating a moderate degree of continuity between the two versions. The comparison of CLASS 2008 and 2nd Edition scores demonstrated significantly different scores on the Classroom Organization Domain, not on Emotional Support or Instructional Support Domain scores. Finally, significant differences were found for the Dimensions of Productivity, Instructional Learning Formats, and Language Modeling (See Table 2) with 2nd Edition scores being higher.

CLASS 2nd Edition was conceptualized as an update to CLASS 2008 rather than a comprehensive revision of the CLASS framework. Thus, it was anticipated that CLASS 2008 and 2nd Edition would be strongly related to each other with relatively minor differences in the average scores between the two versions. Second Edition was revised to include improved coding procedures to increase the reliability of scores and the inclusion of bias recognition and anti-bias strategy implementation; however, the impact of those changes may not be captured and/or reflected in quantitative analyses alone. Discussion will focus on implications for shifting to CLASS 2nd Edition in terms of accountability and supporting systems of improvement.

Authors

• Emily C. Rouge, Teachstone

• Sarah Caverly, Teachstone

• Bridget K. Hamre, Teachstone

• Samatha Mitsven, Teachstone