Evidence in AI: Predictive Algorithms in Healthcare

• In Event: Algorithms at Work: The Practice of Prediction - I: Algorithms-in-Practice in Institutional and Organizational Perspective

Abstract

Artificial Intelligence (AI) is currently applied to big data in the form of machine learning (ML) models in great many domains including healthcare. The all-important goal is to find patterns in data and hereon produce actionable predictions. Decisive to what counts as a valid prediction is the reliability of the labels applied to training data and the thresholds set by ML developers. We argue that we need to critically analyze the evidential bedrock on which predictions are developed and its alignment with the work processes of those applying the predictions and their perceptions of valid, actionable knowledge. Understanding the construction of predictions and the different notions of evidence at play is essential to how we situate algorithmic processes within existing institutional practices and in relation to human decision-making on fragile matters. Based on a case study of the development of a predictive system for public healthcare, this paper examines how negotiations and challenges arise when algorithmic evidence meets the practical interpretations of doctors and thus a profession with a strong background in evidence-based practice. The examination is enabled by ethnographic interviews with general practitioners and ML developers, a theoretical exposition of the concept of evidence, and a literature review on labeling, ground truth and ML thresholds in computer science. The paper illuminates how evidence is a part of the development of AI systems and is situated as a socio-technical practice performed differently across social groups and professions (Bowker & Star, 1999; Shove & Pantzar, 2005; Bijker, Hughes & Pinch, 2012).

Authors

• Anne Henriksen, Aarhus Univeristy

• anja bechmann, UCI & Aarhus University