Multilevel Item Factor Analysis of the Tripod Classroom Environment Survey: Reexamining the Seven Cs

• In Event: 63.018 - Promising Scholarship in Education: Dissertation Fellows and Their Research

Abstract

The Tripod Classroom Environment Survey, a component of the Measuring Effective Teaching (MET) Study, is reported to measure student perceptions of seven theoretical dimensions (7 C’s) of teacher practice and classroom environment, which are in turn hypothesized to be related to student engagement and student outcomes (Tripod Project, n.d.). However, the available empirical evidence has not confirmed that this survey truly measures seven unique dimensions. I will examine the factor structure of the Tripod Classroom Environment Survey using the MET data by applying multilevel item factor analysis methods. Multilevel item factor models are useful in contexts such as student surveys where the group-level characteristic of interest, classroom environment, is measured by individual-level responses, and individual responses are measured by Likert-type items. This methodology allows for an exploration of item and survey properties, such as the dimensionality of the survey, and also provides a means for estimating teacher scores that controls for measurement error and background characteristics of the respondents.

The purpose of the proposed dissertation is to use the large, well-constructed extant Measures of Effective Teaching (MET) dataset, in which careful attention was paid to data collection and management, to determine which and how many dimensions of teacher practice that students can reliably detect using the Tripod survey. I am investigating the Tripod conceptual framework, and will use the methodology of multilevel item factor analysis to examine how well the existing theoretical seven dimensional model is supported by the MET data. Based on the findings of Raudenbush and Jean (2014) and Ferguson (2010), which show strong correlations between some of the factors, it is hypothesized that students’ responses may not allow for the differentiation of the separate factors within the broader teacher support domain. The second central aim of my dissertation is to examine whether accounting for characteristics of students, classrooms, and teachers in the scoring model leads to improved teacher scores that can be used with desired reliability and validity for the intended purposes.

It is hypothesized that the use of a multilevel model that accounts for background covariates with Tripod survey should allow for an analysis framework in which the validity of inferences regarding teacher practice will be improved. When the objective of such surveys is to use data to guide feedback for instructional improvement, failure to control for background factor differences may lead to comparisons of teachers or teaching practices that are not fair and reasonable. It is hoped that this dissertation will contribute to the literature on measuring effective teaching practices by extending advancements in the field of psychometrics to the analysis of student surveys of instructional practice.

Author

• Megan Kuhfeld, University of California - Los Angeles