An integrated approach to assessing for Minimum Proficiency Levels

• In Event: Assessing Foundational Skill Proficiency Linked to International Standards

Proposal

Central to the purpose and development of the Global Proficiency Frameworks (GPFs) for Reading and Mathematics is achieving Sustainable Development Goal 4.1, and specifically indicator 4.1.1:
Proportion of children and young people: (a) in grades 2/3; (b) at the end of primary; and (c) at the end of lower secondary achieving at least a minimum proficiency level in (i) reading and (ii) mathematics, by sex.
The GPFs amongst other instruments have been drawn on to design tools and methods for measuring and promoting attainment of Indicator 4.1.1. An early step was to unpack the terms of Indicator 4.1.1 and define and elaborate the Minimum Proficiency Levels (MPLs) for each of the three stages of schooling. The GPFs were used as key references in this undertaking, with the MPLs (a), (b) and (c) identified as most closely associated with the GPFs’ level descriptions for Grade 2, Grade 5 and Grade 8 respectively.
Fundamental to the unpacking and description of the MPLs were empirically developed Learning Progressions Scales (LPSs) in Reading and Mathematics, using items and data from a wide variety of international, regional and national assessment programs.
A second crucial step leading to the application of MPLs to large-scale assessments was the implementation of an International Standard-setting Exercise (ISSE). Panels that included a range of international partners and regional subject matter experts and practitioners were assembled to determine the thresholds for the MPLs against the LPSs. The bookmark standard-setting method was used in a collaborative virtual exercise conducted over a three-week period across February and March 2022.
The MPL thresholds established by the ISSE are being incorporated into several cross-national studies initiated by the UNESCO Institute of Statistics, beginning in 2022: most recently the Assessment for Minimum Proficiency Levels (AMPL) that, by the end of 2023, will have been implemented in several African countries. AMPL draws on quality-assured assessment items from a variety of sources, and incorporates sufficient material from the ISSE to allow equating to the established MPL thresholds.
A third crucial element in helping countries to monitor progress against Indicator 4.1.1 is the refinement and application of the Pairwise Comparison Methodology (PCM), which allows the MPL thresholds to be mapped to national or other large-scale assessments. PCM is an approach making use of comparative judgement to set global benchmarks on national learning assessments. The PCM approach, again, is based on use of empirical LPSs for Reading and Mathematics.

This contribution to the panel will discuss these integrated activities, showing how they are being applied to help countries and regions in measuring students’ proficiency in Reading and Mathematics against Indicator 4.1.1.

Authors

• Colin Watson, Australian Council for Educational Research UK

• Maurice Walker