Assessment and Accountability Challenges Associated With Competency and Personalized Learning Systems

• In Event: Lessons Learned From the Design and Implementation of Personalized Learning in K–12 Schools

Abstract

Multiple needs and responses have driven the move to personalized and competency-based learning systems, including the desire to enhance the learning outcomes for all students by creating contexts for students to engage in and take control of their own learning. Some might immediately think that such personalization would be a barrier to assessment design. For an extreme example, imagine if every student was pursuing a different learning path. While it might not be very efficient to create a slightly or radically different assessment for each student, as long as the learning targets are explicit, measurement specialists should be able to design appropriate assessments to document what students have learned related to what they were expected to learn. We are able to draw on well-established procedures for assessment design that would not necessarily change for the more personalized case. The real challenge arises, however, when there is a desire or need to include the results from such assessments in school or educator accountability systems.

Ensuring that the rules used to produce the accountability results are fair to the various types of individuals or entities subject to the accountability system is a key tenet of accountability system design. Fairness is manifest, among other ways, by holding people (or schools) to comparable achievement standards using the same or similar types of data, transformations of the data into indicators, and criteria to judge the value of the indicators. More succinctly, “comparability” is viewed by many as an important aspect of designing fair accountability systems. On first glance, personalization and comparability appear to be at odds and, if each is taken strictly, they are. However, in most cases, personalization, at least in K-12 public schools, does not mean complete freedom to choose any possible learning path. In almost all cases, the pace of learning is usually personalized with somewhat less freedom to choose the content to be studied. Nevertheless, even this more limited level of comparability may pose significant accountability challenges.
This paper addresses the range of personalized learning that we might expect in current K-12 systems, but focuses primarily on understanding the ways in which comparability may be considered to help bridge the apparent divide between fair accountability systems and personalized learning. For example, if comparability is defined as strict psychometric interchangeability, it is doubtful that such personalized systems could meet such a threshold. On the other hand, if comparability is evaluated by the ways in which the assessment results predict or support similarly rigorous outcomes, the door may be open to incorporating the results of personalized and competency-based assessments into accountability systems. This paper presents a conceptualization of comparability that is less stringent than interchangeability of student scores. We then present the results of applying such a perspective to a competency-based pilot project where the school-based results must still be used in school accountability determinations. The paper concludes with a discussion of the challenges and opportunities associated with trying to use the results of personalized and competency-based learning systems in large-scale accountability systems.

Authors

• Scott F. Marion, National Center for the Improvement of Educational Assessment, Inc.

• Carla M Evans, University of New Hampshire