Assessing Higher Education Learning Outcomes Globally: A Complement or Substitute to Rankings?

• In Event: 46.017 - University Rankings and the Politics of Higher Education

Abstract

In just a decade, global rankings have become the Billboard Charts of higher education. This paper reviews major international research conducted to help ensure that influential rankings are robust and relevant. It examines what appear as the best means of putting learning and teaching on the map. It shows how the omission of education indicators in most research-oriented rankings constrains their value and distorts system policies and institution strategies. It shows how adding learning and teaching metrics creates and captures important and hitherto untapped insights into higher education institutions and systems.
It has been recognised since their inception that each of these rankings do only partial justice to the breadth of institutional work, particularly regarding education. To be sure, simplifying the dynamically interwoven missions of research, education, engagement and service in ways that facilitate quantitative comparison on an international scale is no small or easy task. Yet without authoritative information on education—particularly for thousands of institutions for which this is the primary function—the rankings carry dangerous potential to distort and homogenise, rather than advance and diversify.
Research shows university rankings meet a growing demand for more transparent information on learning and teaching. Governments and agencies seek such data to steer systems. University executives use this information for leadership and management. Transparent feedback on education also improves teaching. Students and their parents want information to inform their choice of university, and business and industry require better information to forge partnerships and help with graduate recruitment.
Hence there remains a pressing global need to find a cogent and efficient means of adding information on education to prominent university rankings. Such information takes myriad forms, such as survey data on students’ participation in learning, students’ perceptions of teaching quality, marks from tests and exams, enrolment databases records, or peer review of teaching. Various attempts have been made at designing and procuring such information, with perhaps the most prominent example being the OECD’s Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study, conducted from 2009 to 2012.
Taking AHELO as a case, the paper touches on background theoretical and methodological history and literature, including theories of comparative education assessment, policy analysis, and education strategy and practice. Many techniques and concepts were adapted from related studies, but in many instances the adaptation required significant renovation given the unique operating contexts of higher education. Data was collected over a three-year period, using formal meetings, structured consultations, interviews, surveys, and tests, involving 23,000 students and 5,000 faculty at 250 institutions in 17 countries.
The paper argues that assessment in the higher education sector is on the verge of major change, and that this carries immediate implications for institutional strategy, rankings and the geo-politics of higher education. Given the dearth of generalizable information on teaching and learning the expansion of this work carries immediate and potentially radical implications for shaping our understanding of the university quality and positioning.
Background: Hamish holds a Chair of Higher Education, Centre for the Study of Higher Education (CSHE), University of Melbourne.

Author

• Hamish Bennett Coates, University of Melbourne