Incorporating Mixed Item Formats in Computerized Adaptive Testing: A Comparison of Shadow Test and Bin-Structured Approach

• In Event: Computerized Testing: New Developments and Comparability Issues

Abstract

This study investigated: (1). how the mixed CAT works compared with the dichotomous-item-based CAT; (2). how to develop a procedure assembling a CAT with mixed item formats. Two test assembly designs, shadow test and bin-structured approach, were compared under a variety of test conditions. The item pool shape, IRT model used during CAT, and test constrains imposed were manipulated to simulate various real test situations. The results showed that (1). compared with dichotomous-item-based CAT,the mixed-item-based CAT was more efficient but had higher exposure rate; (2). the effects of bin-structured method and shadow test were comparable in test accuracy and content balance, while bin-structure method can improve the item usage and lead to more even distribution of exposure rate.

Authors

• Xin Luo, Northwest Evaluation Association

• Mark D. Reckase, Michigan State University

• Wei He, Northwest Evaluation Association