Equity Beyond Culture? A Sociocultural Critique of the OECD PISA Global Competence Assessment

• In Event: A Critical Analysis of Global Competence and Other 21st Century Skills: Gaps, Fissures, and Erasures within OECD PISA

Proposal

Rhetorically, the Organization for Economic Cooperation and Development (OECD) has emphasized the importance of equity in education. A 2018 OECD report argued, “Achieving greater equity in education is not only a social-justice imperative, it is also a way to use resources more efficiently, and to increase the supply of knowledge and skills that fuel economic growth and promote social cohesion” (p. 4). Equity is thus a central idea presented in their analysis of results from the 2018 PISA Global Competence Assessment (GCA). Along these lines, the OECD found that a key source of difference in access to learning activities related to global competence was the socio-economic profile of students. Based on the 2018 PISA results, they found that socio-economically advantaged students tend to have access to more global learning opportunities than socio-economically disadvantaged students (OECD, 2020b). Furthermore, the report differentiated the 2018 PISA GCA results for students in general and vocational programs, stating that “students in general programs engage more with the conceptual aspects of intercultural learning (e.g. learning about different cultures), while those in vocational programs engage more with the practical aspects (e.g. participating in events celebrating cultural diversity)” (OECD, 2020b, p. 6).

These results indicate an urgent need for a deeper look at questions of equity in relation to the PISA Global Competence Assessment. To address this need, this paper examines the 2018 PISA GCA framework, select test items, and results with a view toward their cultural validity. Drawing on sociocultural and socio-cognitive theoretical perspectives in the science of learning (Solano-Flores, 2019; Oliveri & Mislevy, 2019) as well as other research on the situated and contextual aspects of learning (NASEM, 2018; Darling-Hammond, et. al., 2020), this paper critically analyzes PISA’s construct definition of global competency, the design of select test items, and the presentation of results. We argue that rather than accurately capturing a hierarchy of global knowledge and intellectual ability (with affluent, general program students at the top and lower-income, vocational students at the bottom), the disparities evident in PISA GCA’s results indicate the cultural embeddedness of the OECD’s construct of global competency which, due to its design, inherently favors students from more privileged economic backgrounds.

This analysis offers important insights on the policy-relevance of the 2018 PISA Global Competence Assessment results. This perspective is particularly timely and important due to PISA’s role as an influential global policy lever. The OECD argues that “policy makers, educators and employers clearly need an evidence-based approach to developing and assessing global competence. This is what PISA is about, providing an opportunity to work together across borders to create a better and more humane world” (2020a, p. 6). Yet, in order for PISA to be a more effective tool for equity in global learning opportunities within and beyond OECD member countries, the starting point must be a more inclusive and culturally valid assessment design. Towards this end, PISA test developers should incorporate the latest innovations in evidence-based assessment research drawing on sociocultural and socio-cognitive approaches to large-scale assessment.

Author

• Meg Gardinier, none