Comparison of Human Rater and Automatic Scoring on Students’ Ability Estimation

• In Event: Electronic Board Session I

Abstract

The purpose is to compare human rater with automatic scoring in terms of examinees’ ability estimation with IRT-based rater model. Each speaking item is analyzed with both IRT models without rater-effect and with rater-effects. The effects of different rating design may substantially increase the bias in examinees’ ability estimation.

Authors

• Zhen Wang, Cambium Assessment

• Xiaotong Yang