Exploring the Relationships Between Student Characteristics and Students' Perceptions of Instructional Quality

• In Event: 35.042 - Exploring Student and Teacher Perception Surveys to Learn More About Teachers and Teaching

Abstract

Introduction: Catalyzed in part by initiatives such as the federal Race to the Top competition, states and school districts have developed new teacher evaluation systems employing multiple measures of teacher effectiveness (Hill, Charalambous, & Kraft, 2012). In particular, the use of student perception surveys as a teacher evaluation component has recently accelerated. Burniske and Meibaum (2012) report that as of 2012, eleven states had proposed using student surveys as one measure of teacher effectiveness. School districts such as Memphis and Chicago have already begun incorporating student surveys into teacher evaluations (Schweig, 2013).
The use of student perception surveys as a high-stakes teacher evaluation component has sparked both interest and controversy. On the one hand, proponents argue that students spend hundreds of hours with their teachers each year, affording them a distinctive lens on teacher performance (Peterson, Wahlquist, & Bone, 2000). On the other hand, some researchers have expressed concerns that teachers’ scores may be unfairly influenced by the student demographic composition of their classrooms (Follman, 1992). An additional concern for policymakers is the relationship between student survey scores and achievement outcomes (Ferguson, 2012).

Despite these concerns, however, relatively little research has explored how student demographic characteristics relate to teachers’ scores on student surveys, and how student survey scores relate to achievement outcomes. Using data from the Measures of Effective Teaching Project, Polikoff (2013) found that classroom-level demographics were generally unrelated to teachers’ scores on the TRIPOD student survey. However, Kane, McCaffrey, Miller, and Staiger (2013) found that TRIPOD scores were a significant predictor of student achievement gains in mathematics. To date we can identify few other relevant recent studies.

Research Question: We ask: How are elementary mathematics teachers’ scores on the TRIPOD student survey instrument related to their classroom demographic composition? We also explore how teachers’ TRIPOD scores relate to their value-added scores on both state standardized and researcher-developed assessments.

Methodology: Our sample includes approximately N=246 fourth- and fifth grade teachers from four urban school districts. We collected student demographics and mathematics scores on both state and researcher-developed assessments. Students also completed the TRIPOD survey (e.g. Ferguson, 2012), a student perception survey which has been used and validated in prior research (e.g. Kane, McCaffrey, Miller, & Staiger, 2013).

Results: Preliminary results indicate that teachers’ overall TRIPOD scores are significantly related to the demographic characteristics of students in their classrooms. Teachers with higher proportions of male students received lower average TRIPOD scores. On the other hand, teachers with higher proportions of English language learners or students eligible for a free or reduced price school lunch received higher average TRIPOD ratings. Teachers’ TRIPOD scores were significantly correlated with their value-added scores on both state and researcher-developed assessments.

Brief Conclusion: As the use of student surveys in teacher evaluations accelerates, it is important to investigate how teachers’ scores on these measures are related to the demographic characteristics of their students. We find evidence of significant relationships between classroom demographic composition and TRIPOD scores, as well as between TRIPOD scores and value-added scores.

Authors

• Kathleen Lynch, Harvard University

• Mark Chin, Harvard University