Management Accounting Section Midyear Meeting

Multi-Rater Performance Evaluation and Calibration: Managing Multiple Opinions

• In Event: Performance Evaluation

Abstract

This paper studies the mechanisms and functions of multi-rater performance evaluation and calibration. Specifically, we examine how supervisors use discretion to place different weights on different multi-rater assessments and how calibration committees make decisions to adjust these employee performance evaluations. Using data from an e-commerce company, we document that supervisors use their discretion to place weights on multi-rater assessments consistent with the aim to improve the informativeness of employee performance evaluations by emphasizing (deemphasizing) assessments provided by more (less) informative raters. However, we find that these weighting efforts can be constrained by a high information load placed on the supervisor and the presence of a personal supervisor-employee relationship. Furthermore, we document that calibration committees consider supervisors’ weighting efforts and supervisors’ written argumentation in their decisions to adjust employee performance evaluations and focus their efforts in particular on cases with suspected supervisor bias.

Authors

• Jasmijn Bol, Tulane University

• Maximilian Margolin, RSM Erasmus University

• Daniel Schaupp, WHU-Otto Beisheim School of Management