Working with countries to define ways to report on SDG indicator 4.1.1

• In Event: Data Transparency and Utilization: Re-thinking North-South Dialogue on Learning Outcomes Measurement and SDG Reporting

Proposal

1. RELEVANCE
Assessment systems, including large-scale and classroom assessments and examinations, are an essential means to measure and improve the quality of education. They are important tools for policy- makers and programme implementers in governments to assess and monitor the learning outcomes of students in the country as well as to plan improvements to their education systems.
However, the use of education for evidence-based policy-making is limited, partly due to insufficient institutional capacity of countries that rely on research findings from other countries and regions, even though these findings may not be relevant to their country context or their education systems. It is very much needed recommendation on how to use the data based on current research and practice.
The construction of a scale for reporting in Indicator SDG 4.1.1 is key to facilitate the understanding and use of large-scale assessment. In a very diverse world, the task of agreeing upon a minimal level of competency involves not only technical issues but also political elements. It is a critical piece that integrates The dialogue between North and South came crucial in many dimensions as a way of learning and agreeing on reporting under shared vision with an understanding of teaching and learning.

2. CONTEXT:
The core of the indicator 4.1.1 is measuring the proportion of students at a minimal level of competency. The latent challenge behind this task is to determine what constitutes a minimal level of competency among different education realities worldwide.
There are three political challenges involved in defining a minimum level of competency in literacy and numeracy worldwide. The first relates the level of representation of national curriculum in the definition of the minimal level as most countries have defined learning objectives through their official curriculum.
The second relates to the expected consequences of the assessment. Global dissemination will trigger the social consequence of identifying low achieving countries.
The third challenge concerns external or face validity. Countries with high proportions of the population attending school but achieving below the minimal level of competency could react by challenging the validity of the assessment. Questioning the validity of the test may create doubts about the results within the international community.

3. INQUIRY
The UNESCO Institute of Statistics (UIS) is committed to supporting countries in achieving the goal they set for themselves; “ensuring that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes”. We will present the steps involved in constructing with countries the “UIS reporting scale”:
1) Content standards: what students are expected to learn in reading and mathematics at the three levels of education defined in Indicator 4.1.1 – grades 2/3, end of primary, and end of lower secondary.
2) Performance standards: what students are expected to perform in terms of content.
3) Proficiency scale map(s): how proficiency scales of various NAs and CNAs are aligned with the UIS proficiency scale.
4) performance standards: what students should obtain to be classified into the “desired” performance level for SDG reporting.
5) Improvement Plan and Capacity Development.

4. FINDINGS
A clear vision on how- to facilitate and nurture a global monitoring system of learning that can be used in all countries to produce valid, reliable, and timely data necessary to achieve SDG 4. Not only for global reporting, but first and foremost to support countries to undertake meaningful national policy reforms to improve their education system.
There’s a straightforward association between level of income and participation in regional and international large scale assessments; more than 60% of low income countries don’t participate in any regional or international assessment. Despite the global focus on the most vulnerable countries, Least Developed Countries (LDCs), Land Locked Developing Countries (LLDC), Small Island Developing States (SIDS) and Conflict-affected countries, deserve special attention. They face the greatest challenge in having the financing and technical capacity to facilitate learning assessments.

5. CONTRIBUTION
UIS’s goal, as custodial agency for reporting against the SDGs, is to develop reporting scales that will support national governments to effectively measure and monitor student-learning outcomes in mathematics and reading over time, and to utilize the data for making informed policy decisions. It is a further goal to support the use of existing national assessments (NAs) and cross-national assessments (CNAs) to facilitate measurement and reporting of learning outcomes, rather than requiring a single assessment be used by all countries for SDG reporting purposes.

Author

• Friedrich Huebler, UNESCO Institute for Statistics