Comparing Adolescent and Adult Assessments of Teachers' Classroom Management

• In Event: Contrasting Viewpoints: Using Information From Multiple Reporters to Understand Classroom Dynamics

Abstract

One popular observational instrument to evaluate teaching is the Classroom Learning Assessment Scoring System (CLASS-S; Pianta, Hamre, & Mintz, 2012). Yet, CLASS-S scores poorly discriminate between classroom management quality beyond the identification of the few extremely chaotic classrooms (Kane & Staiger, 2012). Due to this clustering of scores in the most positive range, data generated from these instruments do not adequately support professional development.

This convergent mixed methods study integrated multiple data sources to understand why existing classroom management observational protocols may perform inadequately. The objective of this study is to examine how the student's insider perspective in assessing classroom quality is captured in the observational instruments, almost all of which rely on the outsider's perspective of adult coders. Student self-reports of teachers’ classroom management, adult coder notes and scores from the classroom organization domain of the CLASS-S, and transcripts from adolescent video viewing sessions are compared. The group of participating adolescents included 35 students between the ages of 13 and 15 years old (M = 13.76, SD = .56). The Measures of Effective Teaching (MET) 9th grade class section video library (N = 2,265) provided the potential pool of videos for the video sample. A multivariate multilevel model (using HLM7.0; Raudenbush & Bryk, 2002) simultaneously examined measures of classroom management from two different sources—the CLASS-S and items from the Tripod student perception survey. Using empirical Bayes residuals from this model, scatter plots of CLASS-S and Tripod items were examined to identify concordant and discordant class sections for the cued-video response procedure. The author secured CLASS-S coder notes completed by Educational Testing Service subcontractors. From each sampled video (N = 12), short segments were identified (n = 47) that ranged in length from 1.5 minutes to 3.95 minutes. Sampled videos were shown to adolescent participants during video viewing sessions (56 total sessions across 9 groups of adolescents, with 20% of video segments being watched by two independent groups) using an innovative cued-video response method. This method produced three sources of data—descriptive accounts of the instructional interactions viewed during viewing sessions, verbatim transcripts of the discussions, and time-stamped adolescent commentary. Data analysis consisted of a three-phase analytic process. First, transcripts of the discussions were microanalyzed to assign descriptive codes. A second coding cycle focused on emergent themes. Lastly, adolescent assessments were compared with both CLASS-S coder notes and Tripod student survey results.

Results revealed that outside observers—both adolescents (i.e., video viewers not students in the focal classroom) and adults (i.e., CLASS-S coders)—made assessments of effectiveness of classroom management on the basis of easily observable, low-inference features of interactions focusing consistently on classroom noise level and predictability of teacher behavior. Adolescent outsiders (i.e., video viewers not students in the focal classroom) and adolescent insiders (i.e., Tripod survey respondents in the focal classroom) did not consistently make similar assessments of teachers’ classroom management. Newer conceptualizations of psychological safety as a key feature of well managed learning environments (e.g., Edmondson, 1999; Wanless, 2016) provide an explanatory framework for these results.

Author

• Tanner LeBaron Wallace, University of Pittsburgh