What’s in a Number?: Integrating Machine Learning into a Clinical Context

• In Event: Social Constructions of Data and the Implications of Data Representation

Abstract

Drawing on the existing literature of data-driven practices in clinical care and studies of new technologies within organizations, this paper analyzes how different stakeholders conceptualize, represent, and operationalize a machine learning tool and the data it produces as “evidence” in the course of their existing roles and responsibilities. Our study extends understandings of how data must be interpreted and appropriately represented in order to be integrated into everyday practice. Our analysis is based on ethnographic fieldwork within the emergency department of a large university hospital that is developing, testing, and integrating a tool to assess a patient’s risk of developing sepsis. Sepsis, a syndrome characterized by a body’s extreme response to infection, is difficult to diagnose because its causes and progression are poorly understood from a biomedical perspective.

While machine learning algorithms are often described by critics as black-box systems, healthcare providers also describe the human body as a black box. The issue at stake in both instances is that the causal inner-workings of a system are unknown or unclear. Such ambiguity can foster variation in what is trusted as valid or trustworthy “evidence” across stakeholder communities and the grounds upon which authority is claimed. These communities--and their diverging scripts of action producing validity and certainty--collide in the face of integrating the machine learning tool into clinical contexts. Observed micro-level contestations over evidence and legitimacy provide an opportunity to analyze how a machine learning technology occasions (re)configurations of local authority and professional expertise.

Authors

• Madeleine Elish, Columbia University

• Elizabeth Anne Watkins, Columbia University