Exploring Optimal Cut Scores of an English Language Proficiency Assessment: Receiver Operating Characteristics Analysis

• In Event: Improving Data Use in School Accountability and Education Policy

Abstract

This study applies Receiver Operating Characteristic (ROC) analysis to the English learner (EL) identification context. We explore the ways in which selection of an “optimal” cut score depends on the relative weight given to each kind of classification error (wrongly identifying a student as an EL or wrongly classifying a student as not an EL). This study demonstrates the potential relevance and utility of ROC analysis as method of setting performance standards and cut scores when an external measure is available. The findings of this study have implications for policymakers who set performance standards and cut scores for identifying ELs.

Authors

• Nami Shin, University of California - Los Angeles

• Mark Hansen, University of California - Los Angeles