Measuring Disparate Impact in Human and Machine Decisions

• In Event: Machine Learning and Fairness: Applications in Lending, Benefits, and Clinical Decision-Making

Abstract

Empirical analyses have grown increasingly important in discrimination litigation with the greater availability of detailed data on individuals and decisions. A popular analytic strategy is to estimate disparities after adjusting for observed covariates, typically with a regression model, in hopes of ferreting out discriminatory intent. This approach is, however, ill-suited to auditing algorithms that are now commonly used to aid decisions, which typically do not include race or other legally protected factors as inputs. Moreover, it suffers from two key statistical challenges. First, omitted-variable bias can skew results if the model does not adjust for all relevant factors; second, and conversely, included-variable bias—a lesser-known phenomenon—can skew results if the set of covariates includes irrelevant factors. Motivated by legal understandings of disparate impact, we introduce a new approach, which aims to measure "unjustified" disparities in both human and machine decisions. Our method, which we call risk-adjusted regression, proceeds in three steps. In the first step, we combine all available information in a machine learning model to estimate the value, or inversely, the risk, of taking a certain action, such as approving a loan application or hiring a job candidate. Second, we measure disparities in decisions after adjusting for these risk estimates alone, mitigating the problem of included-variable bias. Finally, in the third step, we assess the sensitivity of results to potential mismeasurement of risk, addressing concerns about omitted-variable bias. To do so, we develop a novel, non-parametric sensitivity analysis that yields tight bounds on the true disparity in terms of the average gap between true and estimated risk—a single interpretable parameter that facilitates credible estimates. We demonstrate this approach on a detailed dataset of 2.2 million police stops of pedestrians in New York City, and show that traditional statistical tests of discrimination can substantially understate the magnitude of (risk-adjusted) racial disparities. As facially neutral decision-making by humans and machines becomes more prevalent, our framework provides a practical, principled method for identifying legally or ethically problematic behavior.

Author

• Johann Demetrio Gaebler, New York University
  Presenting Author