The Use of Automated Scoring to Evaluate the In-Depth Vocabulary Knowledge of English Learners

• In Event: 81.015 - Assessing Language Learning: The Cases of EFL, ESL, and World Language Teaching

Abstract

The purpose of this paper is to (a) test a new automated system developed to score the hand transcribed speech data of second grade Hispanic students, and (b) compare the accuracy of two machine learning techniques. Speech from 217 Hispanic English Learners who were part of a larger study were analyzed using support vector machine (SVM) and tree-based regression (TBR). Findings indicate that the reliability of the automated scoring systems were comparable to human scoring, and that when comparing SVM and TBR, the latter appeared to improve higher Quadratic Weighted Kappa than SVM. Implications of this study are discussed in the context of finding new ways to analyze student natural speech.

Authors

• Doris Luft Baker, Southern Methodist University

• Makoto Sano, Prometric

• Nancy Le

• Marlen Collazo, Southern Methodist University

• Akihito Kamata, Southern Methodist University