Comparisons of Methods of Identifying Unmotivated Test-Takers

• In Event: 47.033 - The Road to Validity Through Student Motivation, (Dis)Engagement, and Aberrant Behavior Detectors

Abstract

There has been growing interest in investigating examinees’ test taking efforts when estimating student level of proficiency. Filtering out unmotivated students can be of great value to test score validity and parameter estimation. To data, several methods have been proposed to account for student test taking efforts. However, few studies have specifically compared the performance of these methods. Given that, this study reviews six motivation-filtering methods and they are compared in terms of the percentage of flagged candidates and the relative frequency distribution of ability (pre/post filtering) using data from a field-test in a computer-based licensure exam. The study has implications to inform research and practice relevant to test score validity.

Authors

• Wei Xu, The American Board of Anesthesiology

• Doyoung Kim, National Council of State Boards of Nursing