Equity Measurement as a “Matter of Concern”: The Case of PISA (Programme for International Student Assessment) in Australia

• In Event: Beyond Evidence: Toward a Politics of Concern

Abstract

Introduction

Inequity has become an intractable problem in education. It presents itself in a variety of ways across different contexts, concentrated around factors such as race, socio-economic status, gender, and immigrant status. Inequity is performed, perpetuated, systematized, addressed and countered in a variety of ways. Improving the equity of school systems has, rightly, become a central goal of most school systems around the world.

In the last decade or so, concern around inequity has led to the widespread measurement of inequity in terms of outcomes. Among these, popular in Australia are PISA’s international comparisons of the equity of school systems. PISA’s quality-equity grid, the comparative ‘slope’ of inequity and the ‘tail of underachievement’ are frequently used in policy and academic circles and are the basis for some of the most significant policy decisions.

This study examines how the abstract notion of equity is rendered precisely measurable and made commensurate across nations. Contrasting the PISA measures with the understandings expressed by policymakers and researchers who highlight the complexity, multidimensionality and messiness of equity, I make a case for moving the conversation on equity from ‘matters of fact’ to ‘matters of concern’ (Latour 2004).

Objectives

This paper aims to trace the systematic simplification that occurs as equity is made measurable and comparable across national borders and interrogates the limits and affordances of such measures. By tracing these processes, it aims to make the concept of equity available for wider public debate so that more effective and more robust solutions may be considered.

Theoretical Framework

Drawing broadly on science and technology studies (STS) and actor-network theory (ANT), this is a sociological study of measurement on the lines of Latour’s laboratory studies. It uses the ANT analytic of translation to trace, step-by-step, the processes used by statisticians as they gradually eliminate uncertainties to produce mobile and decontextualised “policy objects.”

Methodology & Data Sources

Two data sets inform this study. The first comprises interviews with 18 senior Australian bureaucrats, policy advisors, statisticians and academics through 2009, eliciting their understandings of the issues of inequity in Australia collected for a PhD study. The second consists of focused and detailed interviews with a number of Australian and PISA statisticians, and the analysis of PISA documents and of published articles on the subject of equity measurement conducted during 2011 in a follow-up project.

Results & Scientific Significance

By turning the analytical eye to the production of data objects and by problematizing the understandings they generate, this study opens up the ‘black box’ of equity measurement, moves it from the realm of statistical expertise, and makes it available for wider public debate. The use of STS in education is relatively novel, and this study adds to the small but growing number of STS studies in the field.

Latour, B. (2004). "Why Has Critique Run out of Steam? From Matters of Fact to Matters of Concern." Critical Inquiry 30 (Winter 2004): 225-248.

Author

• Radhika Gorur, Victoria University - Victoria Institute for Education Diversity and Lifelong Learning