University Rankings, Public Accountability, and Educational Improvement

• In Event: 46.017 - University Rankings and the Politics of Higher Education

Abstract

This scholarly paper examines the rankings phenomenon through the lens of the broader literature on accountability, and with that background presents an argument for a new approach to performance-related transparency situated in how professions monitor and enforce quality standards.
University rankings are not formal accountability mechanisms, but they share features of accountability systems developed under the “new public management” paradigm, in which the focus of accountability has shifted from answerability for process (e.g., finances and fairness) to results (Behn, 2001). (The specification and measurement of results is problematic in higher education, of course, as it is in many domains of public provision.) To the extent that they are seen by university leaders, prospective students and staff, funders, and policy makers as either authoritative quality assessments or factors that affect access to key fiscal and human resources, rankings command attention and take on the features of a high-stakes test. A well-known characteristic of high-stakes testing is that it induces a focus on the score itself rather than the underlying properties or capacities it is intended to represent—it induces teaching to the test. In the case of rankings, the “test” is the set of metrics on which the ranking is based, and it is taught to by internal decisions intended to move the metrics in a favourable direction. These reactions may reflect either the “acquiesce” or “manipulate” responses set forth by strategic response theory (Oliver, 1991), and they offer an apt illustration of Campbell’s law: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor” (Campbell, 1979, p. 85). (For a related analysis, see Elton’s 2004 treatment of Goodhart’s law in higher education.) This also illustrates the tension between the use of assessment for accountability versus improvement (Ewell, 2009; Findlow, 2008).
The rankings phenomenon is a consequence of a number of broader trends, including massification and the increasing economic rewards attached to higher education credentials. It also reflects the failure of higher education itself to adjust to these phenomena. Massification has led to both a proliferation of providers and a dramatic expansion of those who must choose among a variety of providers. Yet the absence of credible, accessible and comprehensible information about these providers created the market for guidebooks and rankings.
Nearly all university rankings originate from outside the academy, and this is part of the problem. Rather than continuing to complain about methodological shortcomings and deleterious consequences of rankings, higher education must expand its own application of professional norms of peer review and self-regulation to the realm of public information (see Bennett, 2008 and McCormick, 2009) in a way that acknowledges and takes advantage of recent scholarship on transparency mechanisms (see Fung, Graham, and Weil, 2007). The full paper develops these ideas and proposes an alternative to rankings rooted in professional norms and responsibility that attaches priority to educational improvement.

Author

• Alexander C. McCormick, Indiana University - Bloomington