A New Automatic Content Scoring based on Knowledge Graph for Constructed-Response Item

• In Event: AERA Poster Session Wednesday 11:45am
  In Poster Session: Poster Session I -- AERA Division D Section 1: Measurement, Psychometrics, and Assessment

Abstract

Text-based constructed response items are effective in assessing students' higher-order thinking skills. However, achieving standardized automatic scoring of CR-items while balancing rater reliability and ensuring the accuracy of students' ability estimates remains a challenging research task. This study addresses such questions by developing an Automatic Content Scoring method based on knowledge graph representation. It proposes a new algorithm for estimating graph similarity through joint modeling of graph content and structure, and explores the integration of multi-source scoring reference information, including question title information and student response information, to enhance the accuracy of score prediction. This method effectively accomplishes the automatic scoring task for open-ended CR-items in text type, with reliability, validity, and score interpretability all reaching ideal levels.

Authors

• Canxi Cao, University of International Relations

• Liping Yang, Beijing Institute of Educational Supervision and Evaluation

• Tao Xin, Beijing Normal University