Unpacking a global script: theory of change, policies, and instruments of evidence-based policymaking in education

• In Event: Global Education Policy: Evidence, Competencies and Interpretations

Proposal

In the past few decades, we have witnessed a growing emphasis on evidence-based policymaking (EBPM) in education at both global and local levels. However, previous studies have found that the types of evidence used in education policymaking and why and how evidence is used significantly differ across countries (e.g., Sivesind, 2022). Indeed, despite the prevalence of discussion and promotion of evidence-based policymaking, there is a lack of understanding regarding what evidence-based policymaking actually entails and how it is institutionalized in different policy settings. This study addresses this gap by examining how EBPM has been implemented globally with a focus on the theory of change, a bundle of policies, and policy instruments used to advance and strengthen EBPM. It also analyzes how institutional and political factors shape the implementation efforts.

This study draws on a conceptual framework informed by literature on evidence-based policymaking, policy instruments, and policy transfer. The literature on EBPM examines how policy actors use research evidence and data for policy decisions. By examining EBPM, scholars have investigated how and why evidence is used differently across time, policy stages, and national contexts (Baek, 2020; McDonnell & Weatherford, 2013; Steiner-Khamsi & Waldow, 2012). Policy instrument literature focuses on the policy instruments, such as benchmarks and standardized tests, that governments use to formulate policies and how these instruments are shaped by local political and institutional circumstances. Finally, policy transfer literature looks into how global policy ideas are adopted and translated into local contexts and highlights the political aspects of policy transfer drawing on examples of discursive and selective adoption. These three bodies of literature combined offer an interpretive framework to analyze the dissemination and adoption of EBPM in various educational systems across the world.

In particular, this study is inspired by two works that examined the global spread and local translation of neoliberal School Autonomy with Accountability (SAWA) reforms. First, Verger and his colleagues (2019) investigate the reception and evolution of data-intensive governance instruments across the world, using a systematic literature review. They find that while some policy instruments are adopted across geographical boundaries, their application in terms of the intensity of use often depends on specific political and institutional contexts. Second, Steiner-Khamsi and her colleagues (2024) examine whether and how the theory of change, policy instruments, and bundle of policies for SAWA are adopted by comparing two cantons in Switzerland over time. Drawing on policy-relevant documents and expert interviews, they find that the theory of change was not adopted while certain policies and policy instruments were selectively adopted and repurposed based on the Swiss, cantonal contexts.

EBPM has been promoted as a way to enhance educational outcomes and accountability, similar to the SAWA reforms that emphasized data-driven governance. Applying the methodological approaches from the two studies (Steiner-Khamsi et al., 2024; Verger et al., 2019), this study employs a systematic literature review (SLR) drawing from the SCOPUS database to identify the theory of change, policy mix, and policy instruments for EBPM, and examines the potential differentiation across diverse contexts. The SLR involves a systematic search of EBPM-related studies and reports using keywords. Only documents that contain substantive information on the theory of change, policies, and policy instruments for EBPM were included in the final sample. The systematic analysis of the literature finds bundles of policies that facilitate the adoption of EBPM, ranging from evidence collection mechanisms and capacity-building initiatives to expertise-seeking arrangements and legislative mandates requiring use of evidence in policy design and implementation. The findings also identify key policy instruments, such as international large-scale assessments (ILSAs), national assessments, and educational management information systems (EMIS). The findings confirm national variations in how evidence-based policymaking has been adopted across different policy contexts, reflecting the institutional and political circumstances of each country. Preliminary findings as well as their implications will be discussed further in the presentation.

In addressing the CIES 2025 theme, “Envisioning Education in a Digital Society,” the findings of this study have meaningful implications for the digital landscape. As digital technology increasingly affects the experiences of learners and educators, there has been a greater need to develop policies related to digital tools, innovations, and platforms that are “evidence-based.” By critically engaging with the theory of change, policies, and policy instruments of EBPM adopted across different contexts, this study offers insights into how evidence is selected, used, and institutionalized in the formulation of policies that shape the future of education in a digital society.

Author

• Chanwoong Baek, Geneva Graduate Institute & NORRAG