An Investigation of the Impact of Routing Error Under Two-Stage Multistage Testing

• In Event: Roundtable Session 18
  In Roundtable Session: Factors Influencing the Accuracy of Multistage Testing Results

Abstract

The purpose of this study is to investigate the potential impact of misrouting under a two-stage multistage test (MST) design, which includes one routing and three second-stage modules. Simulations are used to create a situation in which a large group of examinees take each of the three possible MST paths (High, Middle, and Low). We compare differences in examinees’ scores associated with different paths under two MST assembly conditions; a small-difference condition in which three modules at Stage 2 overlap in difficulty, and a large-difference condition in which the three modules at Stage 2 are distinct in difficulty. We use two proficiency estimation methods (number-correct scoring vs. pattern-scoring) to determine which scoring method performs well despite misrouting.

Authors

• Sooyeon Kim, Educational Testing Service

• Tim Moses, ETS