Computerized Adaptive Testing With Response Revision: A Model-Based Approach Utilizing All Response Data

• In Event: 29.082 - Roundtable Session 9
  In Roundtable Session: 29.082-12 - Computer Adaptive Testing Designs and Analyses

Abstract

In this paper, we propose a new CAT framework whose ultimate goal is to provide students with as much freedom as possible in revising responses, while preserving the accuracy, efficiency, and validity of the test. Different ability estimation algorithms are considered within this framework and evaluated in different scenarios regarding the test-taking behavior. Our simulation results reveal that when only the most recent response to each item is used for interim ability estimation, students of moderate ability can inflate their scores by employing strategies that take advantage of the response revision feature. On the other hand, ability estimation algorithms that utilize all responses to each item during the test are either robust to such strategies, or penalize them.

Authors

• Shiyu Wang, University of Georgia

• Georgios Fellouris, University of Illinois at Urbana - Champaign

• Hua-Hua Chang, University of Illinois at Urbana-Champaign