Applying Policy Linking Method to Link Assessments to a Global Standard

• In Event: Linking assessments to a global definition of minimum proficiency using policy linking

Proposal

The purpose of this presentation is to describe the procedure of implementing policy linking method to link national and cross-national assessments to a global standard. The implementation of policy linking method follows a three-step process. Step 1 involves defining common content framework for national and cross-national assessments (i.e., knowledge, skills, achievement, and abilities being assessed in the assessments). Step 2 involves describing performance level expectations for the global standard (i.e., what students meeting the global standard should know and be able to do in relation to the common content framework). Step 3 involves setting a cut score between “does not meet” and “meets” the global standard on each national and cross-national assessment.

For Step 1, the UNESCO-IBE (International Bureau of Education) has recently conducted a comprehensive review of 140 national and cross-national assessments. For each assessment, they analyzed the domains, sub-domains, topics, and competencies being assessed. After reviewing all 140 assessments, they developed content frameworks with domains, sub-domains, topics, competencies that are most commonly assessed in national and cross-national assessments. The UNESCO-IBE has defined content frameworks for each grade 2/3, end of primary, and end of secondary levels both in reading and math. These content frameworks are appropriate as a starting point to define the common contents for national and cross-national assessments.

For Step 2, the content frameworks for each grade and subject are used to describe student performance expectations for the global standard in relation to the common contents to ensure consistent student score classification (for meeting the global standard) across countries. A group of subject matter experts (SMEs) including language, curriculum, and pedagogy experts should be invited to a three-day workshop for defining the performance expectations for the global standard. The SMEs should develop the definitions both horizontally and vertically aligned to allow consistency across the grade levels.

For Step 3, the performance expectations for the global standard for each grade and subject are used to establish a cut score between “does not meet” and “meets” the global standard on each national and cross-national assessment using an internationally accepted test centered, student centered, or item mapping standard setting method. For each national and cross-national assessment, a three-day standard setting workshop should be conducted with a group of 10-15 subject matter experts (SMEs). The process of setting cut score should follow a 11-step procedure; the process always begins with training of the SMEs on the performance expectations for the global standard and ends with validation of the process.

The policy linking method has been successfully used in several assessment programs in the USA for linking graduation and collection of evidence tests, GRE and LSAT, ACT and SAT, etc.(Buckendahl & Foley, 2015). The application of this method in the global context for linking assessments with the global standard is unique, robust, and justifiable.


Buckendahl, C. W., & Foley, B. P. (2015). Policy linking as cut score moderation: Considerations for practice. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.

Authors

• Sean Kelly, MSI

• Abdullah Ferdous, MSI