An Exploratory Study of Bias in Estimation of Examinee Proficiency From Self-Tailored Tests

• In Event: Issues in Computerized Adaptive, Computer-Based, and Tailored Testing

Abstract

The purpose of the study was to quantify the difference, or bias, in estimated proficiency from allowing students to self-tailor in-class, undergraduate exams. Data were collected from six educational psychology classes, totaling 262 participating students. On each test, students were permitted to omit from scoring up to five or six items. Rasch ability estimates were generated for each student using both the full set of items and the reduced, or self-tailored set. The weighted mean difference in proficiency estimates was 0.30 SDs; self-tailored proficiency estimates were higher. More than 90% of students who tried self-tailoring raised their scores. Students who did not attempt the self-tailoring option had lower mean estimated proficiency than students who did try self-tailoring.

Authors

• David T. Morse, Mississippi State University

• Jasna Vuk, University of Arkansas for Medical Sciences