Automated Repression: Bias in AI-Assisted Criminal Sentencing in China

• In Event: Surveillance, Repression, and Technology in Autocratic Politics

Abstract

This paper presents the first evidence of systematic ethnic and geographical discrimination in Artificial Intelligence (AI) used to assist judges in criminal sentencing in the People's Republic of China. Using an audit experiment on a commercial criminal sentencing software, I show that, on average, defendants with minority-sounding names and from ethnically diverse regions get predicted sentences that are 5 percent longer than Han defendants who have committed the exact same crimes. Ethnic and geographical factors each contributes independently to this discriminating effect. These findings hold across multiple ethnic groups, regions, and crimes. In addition to the main result, I show that the use of AI introduces new forms of discrimination not seen in human data.

Author

• Eddie Yang, University of California, San Diego