Policy Linking as a Method to Link Assessments

• In Event: Linking assessments to a global definition of minimum proficiency using policy linking

Proposal

The purposes of this presentation are to 1) differentiate between the two types of assessment linking, i.e., statistical linking and policy (non-statistical) linking and 2) examine policy linking as a viable method for reporting to a global standard of student proficiency.

Statistical linking, which includes test equating, is the most widely-used form of linking. It requires building a link between assessments that measure a similar construct using a sample of common persons, i.e., the same students taking different assessments, or a subset of common items, i.e., the same items on different assessments. Once one of these two requirements is met, a psychometrician analyzes assessment data to create a conversion table that shows the equivalence of scores on the different assessments. The conversion table often includes the relationship between the score points on the different assessments and a common scale.

Policy (non-statistical) linking is based on the concept that the linkage from one assessment to another is a set of external criteria, such as performance level descriptors (PLDs) that reflect students’ knowledge and skills by performance category for a subject area and grade level. As in statistical linking, policy linking requires that linked assessments measure a similar construct; however, it only requires the PLDs rather than common persons or common items to build the link. For different assessments, subject matter experts (SMEs) use a common set of PLDs and a standard setting method to make judgments on the difficulty of the assessments. A psychometrician then produces a concordance table that matches the scores on different assessments to each other. With policy linking, only the cut scores, e.g., between passing and failing or proficient and not proficient, on different assessments are often matched to each other. These cut scores can then be linked to a global standard, e.g., for passing or minimum proficiency. Policy linking is less precise than statistical linking, but it is a more practical method in situations where the requirements of statistical linking are not met and/or resources are limited. It is useful in supporting the demand among policy makers for establishing equivalence between multiple assessment pathways that serve the same, or similar, purposes.

Policy linking is a viable method for comparing or aggregating assessments from different programs or countries since its less stringent requirements can be met. Very few, if any, of these assessments, even when they measure the same or similar construct at the same or similar grade level, have common items. Furthermore, the same students do not take the assessments, so creating a data set with common persons would require collecting the data, conducting the statistical linking, and validating the links. Policy linking is much less resource intensive and, hence, more practical in such a situation. While not as accurate as statistical linking, it is precise enough to serve the purpose of allowing education specialists to 1) compare results across assessments and 2) aggregate results from different assessments in relation to a global standard.

Author

• Jeff Davis, MSI