Bridging the Grade Gap: Reducing Assessment Bias in a Multi-Grader Class

• In Event: Education and Replication in Political Methodology

Abstract

Many large survey courses rely on multiple professors or teaching assistants to judge student responses to open-ended questions. While the adoption of, and training on, a grading rubric can help alleviate some of the most extreme versions of bias, there remains the opportunity for students with similar levels of conceptual understanding to receive widely varying assessments. We detail how this can occur, and argue that it is an example of differential item functioning (or interpersonal incomparability), where graders interpret the same possible grading range differently. Using both actual assessment data from a large survey course in Comparative Politics and simulation methods, we show that the bias can be corrected for by a small number of "bridging" observations across graders. We conclude by offering best practices for fair assessment in large survey courses.

Authors

• Sean Kates, New York University

• Tine N. Paulsen, New York University

• Joshua A. Tucker, New York University

• Sidak Yntiso, New York University