Civil-Servant-In-the-Loop

• In Event: Tech Governance: Negotiating Innovation Between the Firm and the State

Abstract

Civil-Servant-In-the-Loop: On the Ethical and Responsible Use of Artificial Intelligence and Machine Learning in Government

From recidivism prediction in policing to screening children by child protective services, the use of machine learning algorithms by civil servants, typically in the form of “risk scoring” systems, has become pervasive. While there has been a great deal of progress made toward defining general rules for the ethical design of artificial intelligence (AI), there is little practical guidance available for ensuring fair and ethical outcomes with the “civil-servant-in-the-loop” in mind. This is especially concerning since the primary use of AI in government currently and for the foreseeable future, will be as a decision-making tool used by civil servants.

Here, we develop a three-part approach to assessing how humans and AI interact in “governance by algorithm”. Indeed, while the complexity of public organizations makes it exceedingly difficult to predict how adopting individual-level rules for handling information produced by artificial intelligence systems will affect outcomes related to fairness, ethics and equality at the organizational level, ensuring that practitioners are aware of the potential pitfalls of making decisions with these systems is undoubtedly a step in the right direction toward ethical governance with artificial intelligence. Accordingly, in this paper we develop a framework for understanding how artificial intelligence can be biased in the context of bureaucratic decision-making and establish a set of guidelines which we argue should be followed, or at least taken into consideration, by individuals at all levels of government when using these systems to make important decisions about citizens.

Specifically, we address the relative roles of human decision-making biases and algorithmic bias, how those biases may interact, and how those biases may be mitigated in the design process. For instance, we know that individual biases of humans in decision-making may be mitigated by “analog” means such as teams (e.g., the “wisdom of crowds”, Delphi method). Likewise, using “explainable AI” (e.g., not deep learning models such as neural nets) increases the likelihood that algorithmic bias can be detected and mitigated. We foresee that such mitigation defenses may be deployed individually or in tandem over time as algorithms come and go (and evolve). Of course, the degree to which such defenses are effective may depend on the role of contractors and vendors in the policy process.

We utilize the working example of recidivism prediction throughout.

Author

• Jason Anastasopoulos, University of Georgia