Isolating Modeling Effects in Offender Risk Assessment

• In Event: Risk Assessment:  Machine Learning vs. Regression—Which Direction Should We Go

Abstract

Objectives Recent evolutions in actuarial research have revealed the potential increased utility of machine learning and data-mining strategies to develop statistical models such as classification/decision-tree analysis and neural networks, which are said to mimic the decision-making of practitioners. The current article compares such actuarial modeling methods with a traditional logistic regression risk-assessment development approach. Methods Utilizing a large purposive sample of Washington State offenders (N= 297,600), the current study examines and compares the predictive validity of the currently used Washington State Static Risk Assessment (SRA) instrument to classification tree analysis/random forest and neural network models.
Results Overall findings varied, being dependent on the outcome of interest, with the best model for each method resulting in AUCs ranging from 0.732 to 0.762. Findings reveal some predictive performance improvements with advanced machine-learning methodologies, yet the logistic regression models demonstrate comparable predictive performance. Conclusions The study concluded that while data-mining techniques hold potential for improvements over traditional methods, regression-based models demonstrate comparable, and often improved, prediction performance with noted parsimony and greater interpretability.

Authors

• Zachary Hamilton, Washington State University

• Melanie-Angela Neuilly, Washington State University

• Stephen Lee, University of Idaho