A New Approach to Analyzing Classroom Observation Data With an Application to Adolescent Literacy Instruction

• In Event: Excellence in Education Research: Early Career Scholars and Their Work

Abstract

In-classroom observational measures of teaching offer the possibility of identifying specific instructional practices that are associated with student outcomes. This possibility has far-reaching implications for educational research and policy, for example by allowing researchers to isolate manipulable, trainable aspects of teaching practice that are associated with student achievement, and by providing teachers and administrators with targeted formative feedback on how to improve student outcomes. However, the instruments commonly used for in-classroom observation (e.g., CLASS, FFT) are scored using simple total scores. Although a total score can tell us how well a teacher performed along a single dimension of teaching effectiveness, it does not encode information about the specific instructional practices that characterize a teacher’s performance. In other words, total scores do not provide diagnostic information, and for this reason they are not ideal for informing research or providing feedback. Moreover, the Measures of Effective Teaching (MET) study has shown that the total scores of many observational instruments have lower reliability and criterion-related validity (with teachers’ value-added) than might be desired. In combination with the inherent loss of diagnostic information, these findings suggest that there is room to improve upon the current scoring methods.
In this research we apply latent class analysis (LCA) as an alternative way of summarizing the observational instruments. We advocate this method because it provides diagnostic information about instructional practices while remaining feasible to implement by the end-user. The basic assumption of LCA is that teachers can be classified into homogeneous (latent) classes based on their instructional practices. The assignment of a teacher to class replaces the total score as a summary measure. Assignment can be done on the basis of a simple table, and therefore is no more tedious to employ than a total score. The diagnostic information provided by LCA (the endorsement probabilities) tell us how likely each class of teachers is to demonstrate the specific instructional practices encoded by the instrument. We refer to this diagnostic information as an “instructional profile.” In intuitive terms, an instructional profile is a multivariate configuration of specific practices that describes what actual teachers are doing in their classrooms.

Author

• Peter F Halpin, New York University