Why do some international assessments succeed, and others do not?

• In Event: The OECD’s Defining Role in Education: Its Historical Rise, Global Impact and Comparative Perspectives

Proposal

International large-scale assessments (ILSAs) are considered a success story. The past two decades witnessed an immense growth in the number of international and regional assessments and in country participation in these assessments (Kamens & McNeely, 2009). Scholars disagree on the relationship between country performance on ILSAs and economic development, which it the main socio-logic behind the ILSAs regime (Hanushek & Woessmann, 2010; Ramirez et. al, 2006). Never-theless, scholars have documented the ways in which these assessments shape public discourse, public opinion, and educational policy (Green Saraisky, 2015; Pizmony-Levy, forthcoming; Pizmony-Levy & Bjorklund, 2016; Waldow, Takayama, & Sung, 2014).
This perspective on ILSAs, however, is not complete as it focuses on successful cases. That is, cur-rent scholarship on ILSAs overlooks failing attempts to launch ILSAs. Drawing on organizational the-ories and innovations studies (Binder, 2000; Fullan, 2001; Rogers, 2010), I seek to address the fol-lowing research question: Why do some international assessments succeed, and others do not? In other words, I examine the extent to which different factors explain successful institutionalization of ILSAs (i.e., long term and sustainable administration).
I compare three organizations that initiated and administrated ILSAs: (1) the US-based Education Testing Service (ETS) and its International Assessment of Educational Progress (IAEP); (2) the Interna-tional Association for the Evaluation of Educational Achievement (IEA) and its Trends in Internation-al Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS); and (3) the Organisation for Economic Co-operation and Development (OECD) and its Pro-gram for International Student Assessment (PISA). Although these three organizations have offered similar technical solution to the growing demand for achievement data in key subjects, they achieved different outcomes with respect to sustainability. While assessments by the IEA and the OECD were institutionalized into regular cycles, the ETS-IAEP was abended after two cycles.
Based on archival research and interviews, I find that two factors have contributed to the failure of the ETS-IAEP. The first factor is “buy-in” from participating countries. In contrast to assessments by IEA and OECD, ETS was not able to recruit diverse set of countries to participate in its first two rounds of IAEP. The second, and related, factor is “international optics.” Both IEA and OECD are international organizations that bring together representatives from different countries. The ETS is a US-based non-for-profit that sought to export the National Assessment of Educational Progress (NAEP) to the world. Lacking the international optics and legitimacy, ETS-IAEP failed to institutional-ize. In the full paper I discuss the consequence of this outcome and how it affected the ILSAs field and the specifically the IEA.

Author

• Oren Pizmony-Levy, Teachers College, Columbia University