Application of Item Response Theory Model for Continuous Response in the Scholarship Evaluation System

• In Event: Virtual Poster Session
  In Virtual iPresentation Only: SIG-Research on Evaluation Virtual Poster Session

Abstract

The subjective evaluation system for educational fairness presents significant challenges. This study proposes applying the beta-IRT model to the Chinese National Scholarship evaluation, considering judges' severity, discrimination parameters, and consistency, to accurately assess students' latent abilities. Using Markov Chain Monte Carlo (MCMC) algorithms within a Bayesian inference framework, the study addresses the issue of small sample sizes. Results show that the model fits well, with rankings based on the model highly correlated with those from traditional methods, though with lower consistency. This indicates that while trends in rankings align, specific placements may vary, suggesting that different ranking methods could impact scholarship distribution. Additionally, stricter judges tend to have lower discrimination ability and consistency, which should be noted in future evaluations.

Authors

• Xuanyi Cai, Beijing University of Chinese Medicine

• Yixing Liu, Beijing University of Chinese Medicine