In a Fair Possible World: Evaluating Measures of Algorithmic Fairness with Counterfactual Queries

• In Event: Constellation Thinking and AI Ethics

Abstract

Machine learning algorithms are playing an increasingly prominent role in the distribution of social benefits and burdens. While the mathematical appearance of algorithmic decisions gives the illusion of neutrality, there are increasing worries that automation has disproportionately harmed society’s most vulnerable groups. In response to such concerns, there has been a burgeoning line of research devoted to the construction of various metrics for quantifying potential algorithmic unfairness. Yet this proliferation of quantitative fairness measures is problematic, in that different measures often produce divergent verdicts. This talk provides a much-needed methodological constraint on theorizing about measures of fairness by examining the underlying normative principles for which these measures are meant to be a proxy. In particular, it will be argued that any adequate measure of fairness must be capable of accurately answering a set of counterfactual queries that are entailed by claims about the potential moral wrongness of a given decision. Aside from its consequences for the debate generated by distinct measures of fairness, the proposed constraint also provides a means for parsing responsibility in cases of algorithmic harm. 

Author

• Sina Fazelpour, University of British Columbia