A Philosophy for the Science of Explainable AI

• In Event: Politics, Ethics and AI Technology: The State of the Discipline

Abstract

This paper reviews recent efforts to develop a rigorous science of explainable AI (or XAI, as it is sometimes called), evaluates these in light of recent philosophy of science and discusses promising ways forward for this emerging field of research.

A common objection to relying on AI systems in ethically sensitive domains is that it can be difficult to adequately explain their decision-making to humans. AI systems, especially those based on advanced machine learning techniques, are often accused of being “opaque”, “black boxes”, “uninterpretable” or “incomprehensible”.

In response, a growing field of research seeking to develop techniques for making AI systems more explainable has emerged. While early work in this field tended to rely on researchers’ intuitions of whether something constitutes an “explainable” or “interpretable” system, AI researchers are increasingly calling for more rigorous approaches to AI explainability (Lipton 2016; Weller 2017; Doshi-Velez and Kim 2017; Miller 2018). Two general types of approaches have been proposed: First, empirical approaches which seek to devise experimental methods for measuring how explainable a system is, e.g. by measuring representative users’ feeling of comprehension or performance on domain-relevant tasks. Second, theoretical approaches which seeks to design AI systems based on existing accounts of explanation in social science, psychology or philosophy.

Both approaches represent valuable steps forward. However, this paper seeks to highlight some of their remaining limitations. Drawing on recent philosophy of science, I argue that adequate explanation is a contextual, pragmatic and value-laden phenomenon. There are many different explanations that can be given of the same system and which is most relevant varies by context. An adequate explanation supplies its audience with information which enables them to successfully take actions or make inferences which are deemed contextually valuable. Consequently, there is no meaningful, context-independent answer to whether a given system is sufficiently explainable or interpretable to allay the objections to relying on AI systems in ethically sensitive domains.

Instead, I propose that progress in the development of adequately explainable AI will require the development of context-sensitive “mid-level” theories which specify what kinds of explanations are appropriate to a given domain of application. The paper concludes by outlining what will be involved in developing such theories. Rather than starting from general theories or measures of explanation, these will start from normative accounts of what kinds of actions and inferences are important for different stakeholders in a given domain. Based on these, researchers should then seek to determine (a) what information these stakeholders need in order to make these inferences or actions successful and (b) what kinds of explanations will best supply this information.

Author

• Rune Nyrup, University of Cambridge