A Meta-Analysis of the Relation Between Automated and Human Scoring in Essay Writing

• In Event: 14.019 - Advances in Writing Instruction, Assessment, and Interventions

Abstract

The purpose of the study is to examine the degree of correlations between automated and human scoring, and the relation between the correlations and essay writing features. For this meta- analytic study, 99 studies with the total sample size of 465,981 are derived from 23 study sources. Correlation between human and automatic scoring are not consistent across studies, but the average correlation is strong and positive. Different kinds of automated essay scoring systems provide different correlations between automated and human scoring. The correlations do not change by the number of human raters. The automated essay scoring system has the advantages of reducing the time and resources consumed when the number of the writings is excessive in large scale assessments.

Author

• Jiyeo Yun, Florida State University