Using National Assessments data and the revised Policy Linking procedures to report on SDG 4.1.1

• In Event: Highlighted Session: Reporting on SDG 4.1.1: A critical examination of the costs and benefits of emerging options

Proposal

The policy linking methodology was developed to allow countries to use their existing – sub-national, national, and cross-national –assessments to report against Sustainable Development Goal (SDG) 4.1.1. Policy linking makes use of a standard-setting methodology to set benchmarks on learning assessments using the Global Proficiency Framework (GPF) which provides a common reference for reporting progress on indicator 4.1.1 of the SDGs.

The policy linking methodology is elaborated in the Policy Linking Toolkit, which provides guidance and templates to countries, donors, and partners who conduct policy linking workshops to set global benchmarks. The toolkit and the accompanying Quality Assurance Policy specify the steps to be taken before, during, and following the workshops to ensure consistency and, as a result, comparability of the outcomes. After two years of piloting in several countries, among which India, Bangladesh, and Nigeria, the toolkit was updated. After the revision, another round of piloting took place in at least five countries using national assessments.

The piloting demonstrates the flexibility of the toolkit. The policy linking has been performed in different countries, with different national assessments, for different grades, and with different designs. The policy linking workshops have been performed in-person, remote and in a hybrid fashion. Even though the current toolkit does not contain guidelines for assessments analyzed with IRT, during the piloting several countries participated with assessments that were analyzed using Item Response Theory (IRT). IRT requires expert psychometricians and as such, it is a human capital challenge for countries.

Policy linking carries many benefits. Reporting on SDG 4.1.1 using national assessments, rather than international or regional assessments, is attractive for countries. As existing data and existing assessments are used, the financial costs for countries remain moderate. Furthermore, the fit of the national assessments with the country’s curriculum and content frameworks is perceived as an important benefit for countries.

A country-driven approach like policy linking might be less efficient than international approaches, but it is an important tool for capacity building and empowers countries. In this sense, the policy linking carries important equity benefits. A stronger focus on capacity building (including training and manuals) increases the costs, but also enhances the comparability and equity.

The flexibility of the approach comes also with a cost. The comparability of the statistics across countries is limited in comparison with international and regional large-scale assessments. The focus at the moment is on formulating additional psychometric or methodological guidelines for analyzing the workshop outcomes. These guidelines should improve the comparability of each learning assessment program and each country’s national statistics over time.

Author

• Sanneke Schouwstra, CITO