Quantifying Miscalibration: Estimating the Impact of Demographic Differences on LS/CMI and ORAS Accuracy

• In Event: Rethinking Risk: Addressing Bias, Accuracy, and Adaptability in Criminal Justice Risk-Need Assessments

Abstract

Risk assessments such as the LS/CMI and ORAS are central to decision-making in criminal justice systems across the United States. Despite their widespread use, prior research suggests that these tools often lose predictive accuracy when applied in jurisdictions whose demographics differ from those of the original development samples. While this phenomenon raises concerns about fairness and effectiveness, the exact extent of its impact remains unclear. This study examines risk assessment data from six states and addresses the gap by matching our sample’s demographics to those used in developing the tools. By aligning the sample’s demographics with those used in LS/CMI and ORAS development samples, I isolate the effects of demographic differences on predictive performance. Through comparison of matched and unmatched samples, the findings estimate the level of miscalibration that local agencies can expect from contemporary assessments based on the characteristics of their jurisdiction. Ultimately, this project aims to improve the accuracy and equity of risk assessments, supporting more informed and fair decision-making in diverse criminal justice settings.

Author

• John Michael Ursino, University of Nebraska Omaha