Students’ Achievement of Reading and Math Proficiency for SDG 4 on Provincial and National Assessments

• In Event: Progress on Measuring SDG 4 – Improved Methodology, Community of Practice, and New Results for Policy Linking on SDG Indicator 4.1.1.

Proposal

The primary results for the policy linking workshops at the provincial and national levels consisted of the quantitative results (i.e., benchmarks, impact data, and consistency statistics) and AIR’s recommendations about policy linking and the assessment for the donor and government.
Quantitative Results. The policy linking workshops produced final benchmarks, i.e., the raw scores that delineate the four GPLs of 1. Does not meet minimum proficiency, 2. Partially meets minimum proficiency, 3. Meets minimum proficiency, and 4. Exceeds minimum proficiency.
The benchmarks are measured in raw score points for the subjects of the national and sub-national assessment. The impact data which are the proportion of students in each GPL are derived from applying the benchmarks to the student-level assessment data provided by the government to AIR. The proportion of students who meet minimum proficiency is the sum of the meets and exceeds GPLs. The percentages of students in each GPLs are disaggregated by gender and province.
There are three consistency statistics. The standard error of the benchmark measures the consistency of panelists’ benchmarks in each subject. A value of 1.00 or less is considered acceptable, though it depends on the number of items on the assessments, with more items showing higher standard errors (Authors, 2020). The inter-rater consistency is a statistic that measures the agreement for the panel on item ratings by considering all pairs of panelists. The intra-rater consistency is a statistic used to evaluate how well each panelists determined the item difficulty of their assessments. Values of 0.80 or more are tentatively considered as acceptable (Authors, 2020).
Policy Recommendations. The workshops produced five main recommendations. First, an overall recommendation is that the government should assess key learning standards in specific school grades with certain periodicity, including early years to grade 4 as well as lower secondary. Second, the government should consider the content standards in the GPF for the development of future iterations of the assessment so that their assessments are more aligned with global content. Third, the government can hold workshops on the alignment of the GPF’s content standards and the assessment, as well as materials and training (i.e., following guidance from the Minimum Proficiency Review or MPR, that is currently under development by donor agencies) with the goal of increasing system efficiency and learning outcomes. Fourth, the government can use the GPF’s performance standards in item development and test construction to ensure that the assessment measures learning at the right level and interprets the assessment results in relation to the performance standards. Finally, the government should continue to apply the benchmarks to future assessment results, with test equating from one administration to another, and use the benchmarks to track student performance over time.

Authors

• Abdullah A Ferdous, American Institutes for Research

• Jeff Davis, American Institutes for Research