Studying Effects of Data Collection Designs on Online Testing Comparability Studies

• In Event: Validity Investigations

Abstract

Traditional data collection designs –single group counterbalanced and random groups– champion test mode effects research on computer-based assessment. This paper examines them and two alternative data collection designs under a condition of measurement free from test mode effects and samples of participants from a common population familiar with computers. The experimental data included a population of examinees, four data collection designs, two test modes of a target assessment, and an anchor assessment. Test-level results supported the superiority of the single group counterbalanced design and the random groups design relative to the other designs. The promise of valid use and interpretation of test scores from computer testing relies, in part, on the adequate selection of data collection designs.

Authors

• Alvaro J. Arce, Pearson

• Ileana Diaz-Cervera, University of Yucatan