Reusing Mixed-Format Tests: Can We Adjust for Scoring Shift Without Rescoring Previous Constructed Responses?

• In Event: Roundtable Session 6
  In Roundtable Session: Research on Rater and Scoring Issues

Abstract

This study compared two ways of dealing with a problem that can occur when a mixed -format (MC and CR) test form is re-administered: a change in the CR scoring standards. The only way to detect if the scoring standards have changed is to have raters at the new administration rescore CR responses from the previous administration. The purpose of the study was to see if anchor equating through the MC portion of the test could substitute for rescoring a sample of the CR responses from a previous administration. The study showed mixed results. Based on the results, we cannot recommend the use of anchor equating through the MC portion as a substitute for equating in a rescoring sample.

Authors

• Chi-Wen Liao, Educational Testing Service

• Wei Wang, Educational Testing Service

• Yi Cao, Educational Testing Service