A Meta-Analytic Review of the Predictors of Student Evaluations of College Instructors

• In Event: Meta-Analytic Methods and Concepts at Crossroads: Relations of Student and Context Variables With Meaningful Outcomes

Abstract

College students’ ratings of instructors are widely used to provide instructors with feedback on performance, inform judgments about the quality of instructional methods, and are even used to inform hiring, promotion, and contract renewal decisions. Although the use of student ratings is almost ubiquitous in US colleges and universities there is widespread controversy about their validity. Some (e.g., Marsh, 1982; Marsh & Roche, 1997) have found evidence for the validity of students’ evaluations of instructor performance and advocate for their use – in part because ratings by peers and supervisors largely reflect maximal performance while student ratings are better indicators of the typical performance of instructors. Many others (e.g., Ambady & Rosenthal, 1993; Williams and Ceci, 1997) have expressed concern about the meaning and validity of student ratings; concerns that appear reasonable when considering findings from the performance appraisal literature (e.g., Fletcher & Perry, 2001; Latham & Mann, 2006) which notes that untrained raters (like college students) are susceptible to rating errors such as halo error and contrast effects. This would suggest that student ratings should not be weighed heavily when making high-stakes decisions about the employment conditions of instructors. Similarly, researchers have noted that students’ ratings of their instructors appear to sometimes by meaningfully influenced by or related to their expected grades in the class, their actual grades, and various other, theoretically performance-irrelevant characteristics of either the instructor (e.g., gender, attractiveness) or the class (e.g., difficulty, frequency of classes).

In this paper we provide a meta-analytic synthesis of the large literature on the relationship between college students’ ratings of their instructors and the characteristics of students, instructors, and classes. We rely on the Hunter and Schmidt (2004) meta-analytic method and examine our data at two levels. First, we examine those characteristics that are associated with the ratings given by individual students. Second, we examine those characteristics that are associated with the average class ratings received by instructors. Our results, based on 337 unique samples from over 400,000 students and over 10,000 instructors indicate that college students’ evaluations of their instructors are moderately related to a number of student and class variables that are theoretically largely unrelated to instructor effectiveness. We will discuss the relationship between overall instructor ratings and variables reflecting the instructor (e.g., instructor gender and age), the class itself (e.g. time of time, class size, class content); students’ expected and actual grades, and characteristics of the student (e.g., student personality, GPA, admissions test scores, gender). Some of the variables that are related to students’ evaluations of their instructors may also be related to actual teaching effectiveness but many of the observed relationships and the relative lack of experimental evidence relating to the validity of student ratings suggest that institutions should be cautious about relying on student evaluations of instructors as valid indicators of instructor performance. We will discuss the general validity of student ratings of instructors as well as potential unintended consequences of relying on student ratings of instructors.

Authors

• Marcus Crede, University at Albany - SUNY

• Nikki Blacksmith, George Washington University