Academic Evaluation as Signal and Symbol

• In Event: Expertise, Assessment and the Effectiveness of Research

Abstract

The number of studies claims that the use of quantitative metrics in academic evaluation is linked to the state desire to cut funding for public higher education. When governments invest in higher education they need some grounds for decision-making. Although metrics are clearly related to state’s desire to emphasize the higher education’s economic role, this assumption has rarely tested in an empirical way. Data from multiple resources on state control of Russian universities allows us to clarify how the state uses university’s evaluations for decisions to punish the worst. All Russian universities are obliged to participate in an annual evaluation organized by the Ministry of Education and Science. As a result, the Ministry has information about dozens of metrics for hundreds of universities. Within the classical decision-theory point of view, information is gathered because of the desire to improve decisions. Empirical research on using evaluations in the case of state decisions about universities demonstrates a pattern that can’t be explained in such terms. Particularly the state systematically penalizes the universities which are simultaneously evaluated as effective organizations. Instead of using different kinds of metrics about research and educational activity the state bases their decisions on the university type. The severe sanctions including the shutdown the university depend on whether the university is a private organization. We suggest that this finding is a consequence of bureaucratic control of academic organizations which operates while being distanced both physically and symbolically. Fine-graded metrics concedes simple categorization because their application makes the decision-making more complicated and questionable. There is no an agreement on the key performance metrics for academic organizations. From the point of view of state control, it is natural to suspect private organizations to be the main focus of scrutiny even if their performance results are passable. We observe a situation where the state asks for evaluation reports but evidently does not read it. Considering these phenomena we conclude that these reports are to legitimize decisions by symbolizing a commitment to a rational decision-making (Feldman, March 1981). This study uses data on the application of sanctions in relation to universities (cancellation of a license and accreditation). The data about sanctions are merged with results of annual evaluations of universities conducted by the Ministry in 2015 and 2016 (1,748 cases).

Author

• kateria Guba, European University at St.Petersburg