The Impact of Competing Events on the AUC in External Validation Studies

• In Event: Advances in Risk Assessment and Prediction

Abstract

The ability of a risk scale to accurately discriminate cases from controls in new data is tested by conducting an external validation study. These studies are sometimes called predictive validity studies. Various kinds of events of interest are used as criteria in such studies depending on the risk scale examined and the application. Typical events of interest in criminal justice settings include arrest, violent arrest, and supervision failure. Along with the event of interest, competing events may be present. For example, a researcher might test the ability of a risk scale to discriminate persons arrested for a violent offense within three years of prison release; however, some persons in the sample were arrested for a non-violent offense and incarcerated during the follow-up period. The approach in most external validation studies in criminology is to ignore competing events, treating persons that experience a competing event as a success (control). A common measure of discrimination used in external validation studies is the area under the receiver operating characteristic curve (AUC). We demonstrate the effect of using different methods and models on the estimate of the AUC in the presence of competing events.

Authors

• William Dieterich, Equivant

• Christina Mendoza, Equivant

• Tim Brennan, Northpointe Institute