Poster #29 - Pairwise Comparison Using an Adaptive Selection Algorithm: Efficient Holistic Measurement

• In Event: 79.052 - MTCC Poster Session 19
  In Poster Session: 79.052-2 - Educational Measurement, Psychometrics, and Assessment Poster Session

Abstract

Pairwise comparison is becoming increasingly popular as a holistic measurement method in education. Unfortunately, many comparisons are required for reliable measurement. To reduce the number of required comparisons, we developed an Adaptive Selection Algorithm (ASA) that selects the most informative comparisons while taking the uncertainty of the object parameters into account. The results of the simulation study showed that, given the number of comparisons, the ASA resulted in smaller standard errors of object parameters than a random selection algorithm that served as a benchmark. Rank order accuracy and reliability were similar for the two algorithms. Caution is required for interpreting the Scale Separation Reliability when the ASA is used, because this coefficient may overestimate the benchmark reliability.

Authors

• Elise Crompvoets, Tilburg University

• Anton Beguin, Cito

• Klaas Sijtsma, Tilburg University