Prioritizing Homelessness Program Access via Algorithm vs. Human Judgment

• In Event: Strengthening Rapid Re-Housing Programs

Abstract

This paper compares how well human judgment versus algorithmic models perform in prioritizing access to scarce homelessness programs. In the United States, communities must ration limited housing subsidies through coordinated entry systems that rely on prioritization rankings. Historically, many communities have used tools like the VI-SPDAT, which have faced criticism for exhibiting poor predictive validity regarding future risk. Working closely with the homelessness services system in Bexar County, Texas (San Antonio), we designed and implemented a randomized controlled experiment to compare two novel prioritization rankings side-by-side. The first system is a data-driven algorithm built using existing administrative data from the Homeless Management Information System (HMIS). The second system relies on the subjective ratings of human experts who staff the intake system. Assessors were prompted to predict a client's vulnerability to continued homelessness if they did not receive housing assistance. To mitigate the incentive for assessors to artificially inflate scores to secure aid for their clients, the human scores were re-scaled to use only the relative rankings of clients within a given assessor's caseload. Both the algorithmic and human systems were designed to predict the same outcome: the baseline risk of persistent homelessness among a group of clients, measured as a new enrollment in homelessness programs 6 to 24 months after their initial assessment.

Our analysis yields two main findings. First, the data-driven algorithm predicts the persistence of homelessness substantially more accurately than both the status quo VI-SPDAT tool and the human assessments. Algorithmic scores do predict risk of persisting in homelessness for a sample of people requesting services after the time period used to train the algorithm. On the other hand, while the re-scaled human scores exhibit some predictive power, this is largely due to benchmarking against the algorithm's risk profile across different assessors. Second, the two systems inherently prioritize different populations. The data-driven algorithm systematically emphasizes clients with extensive histories of homelessness, older age, and significant behavioral and physical health issues, including mental illness, substance abuse, and physical disabilities. Conversely, human assessors tend to prioritize families with children and individuals reporting lower incomes.

Author

• David C Phillips, University of Notre Dame
  Presenting Author