Making PISA Fit and Making the World Fit PISA: the OECD's Domestication of Low-Income Nations

• In Event: Discourses of International Organizations in Education – What Do They Promote, When and How

Proposal

With the administration of PISA over the last twenty years, the OECD has become one of the most influential actors in education policy globally. Although designed for the rich, market economies of the OECD, many non-OECD nations – including 26 middle-income and some low-income nations – have participated one or more times in PISA. However, a major OECD report suggests that the regular PISA is not suited to the policy needs of middle- and low-income nations. OECD has therefore developed a new version of PISA – PISA for Development (PISA-D) – that aims to be more suited to the contexts and the policy needs of middle- and low-income nations. With PISA-D, the OECD seeks to position itself as a key actor in the benchmarking of Sustainable Development Goal 4.
This paper examines how PISA is being adapted to become a good ‘fit’ in middle- and low-income nations, based on empirical research in PISA-D countries, including observation and interviews with government officials, PISA contractors and OECD officials, as well as analysis of PISA-D working documents and reports. We find that while PISA-D requires the adaptation of PISA to fit ‘low-income contexts’, these ‘contexts’ are also adapted to ‘fit’ PISA. Changes to PISA include adapting the cognitive questionnaire to include more questions in levels 1 and 2; making the background questionnaires on socio-cultural advantage more relevant to communities surveyed; changing sampling techniques and the methods of presenting PISA to situations where a significant percentage of 15-year-olds are out of school, and so on. At the same time, OECD requires nations to undergo a series of profound and systemic adaptations in order to be ‘fit for PISA’. These include the development of stakeholder interest in PISA-D and in using data; cooperation between stakeholders; transparency and autonomy in the relevant government departments; government support for the development of assessment capacity within nations; commitment to institutionalising an assessment culture and a culture of data use, and so on. To assess and develop these capacities, extensive interviews, workshops and consultations are undertaken with a range of stakeholders. Making these nations ‘PISA-fit’ could also include acquiring physical resources and changing the IT infrastructure and data storage practices. Thus nations are ‘domesticated’ through significant changes even before PISA is conducted in these nations. While the effects of PISA results on national policies has been widely explored, this paper explores the ways in which OECD influences nations in profound ways even before the assessment is conducted.
Our research takes a socio-technical approach and is guided by concepts from Science and Technology Studies. In particular, we employ Callon et al’s conceptualisation of the three translations in research: simplifying entities to become fit for the laboratory; manipulating the simplified entities in the seclusion of the laboratory; and then laboratorising the world so that the entities from the secluded laboratory can survive ‘in the wild’.
This paper highlights the socio-technical nature of power and extends the literature on policy transfer and the globalisation of policies and practices.

Authors

• Camilla Addey, Teachers College, Columbia University

• Radhika Gorur, Deakin University