Does Vertical Scaling Based on Content or Distribution Type Influence the Classification Decisions on Annual Yearly Progress Measures?

• In Event: Roundtable Session 18
  In Roundtable Session: Assessing Student Growth: Methods, Issues, and Applications

Abstract

Vertical scaling establishes a score scale across grade levels. This study evaluated four vertical scaling procedures (concurrent calibration, fixed common item parameters, test characteristic curve, and hybrid characteristic curve), across two content areas (Reading and Mathematics), and two score distribution types (normal and negatively skewed). Two outcome measures were used: decision consistency (DC) and decision accuracy (DA) at two cut-scores. The DC results were consistent across vertical scaling methods, content areas, and distributions (>0.80). The DA results varied depending on the grade, scaling procedure, content area, and distribution. Practitioners should evaluate the DA results across these conditions when creating scales to ensure that the classification results are as accurate as possible as these are typically used for annual yearly progress.

Authors

• Andrea Gotzmann, Medical Council of Canada

• Mark J. Gierl, University of Alberta

• W. Todd Rogers, University of Alberta