Poster #37 - Evaluating the Accuracy of Predictors in Early Warning Indicator Research: Receiver Operating Characteristic (Area Under Curve)

• In Event: 77.059 - MTCC Tuesday Poster Session 18
  In Poster Session: 77.059-5 - Evaluating the Accuracy of Predictors in Early Warning Indicator Research: Receiver Operating Characteristic (Area Under Curve)

Abstract

Early Warning Systems (EWS) and Early Warning Indictors (EWI) have emerged as an attractive domain for states and districts interested in predicting student outcomes using data that schools already collect with the intention to better time and tailor interventions. However, current diagnostic measures used across the domain do not consider the dual issues of sensitivity and specificity of predictors, key components for considering accuracy. We apply signal detection theory using Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) analysis. Using nationally generalizable data from the Education Longitudinal Study of 2002 (ELS:2002) we provide examples of applying ROC accuracy analysis to a variety of predictors of student outcomes including high school dropout, college enrollment, and postsecondary STEM degrees and careers.

Authors

• Alex J. Bowers, Teachers College, Columbia University

• Xiaoliang Zhou, Teachers College, Columbia University