Can AI Reduce Caseworker Burdens and Improve Caseworker Performance?

• In Event: Automation, AI and Nudges: Improving Digital Interfaces to Reduce Administrative Burdens in the Safety Net

Abstract

This study evaluates whether generative AI can enhance frontline service delivery by supporting benefit navigators—staff who assist clients in understanding and applying for programs like SNAP. Specifically, we examine the effectiveness of a chatbot developed by the civic tech nonprofit Nava, which generates policy-aligned suggestions based on federal and state benefit manuals. Our research questions are: Does access to the chatbot improve the accuracy of navigator responses? Does it reduce disparities in performance across question types and client profiles? Can it help alleviate navigators’ perceived administrative burden and improve response efficiency?

Our research builds on the theory of administrative burden, which conceptualizes the barriers in interacting with government systems as learning, compliance, and psychological costs (Herd & Moynihan, 2019). While most research focuses on burdens experienced by benefit applicants, we extend this framework to frontline workers, exploring whether AI tools can reduce learning costs for benefit navigators. We also draw on the literature on human-AI collaboration (Green & Chen, 2019), which explores how people engage with algorithmic suggestions and the conditions under which AI enhances or distorts human judgment. Finally, we engage with the literature on algorithmic equity (Eubanks, 2018; Obermeyer et al., 2019) to examine whether generative AI tools promote fairness or risk reinforcing disparities in service delivery across different client profiles.

We conducted an offline survey experiment designed to simulate real-world decision-making by benefit navigators in a controlled, low-risk environment, as part of a research collaboration with two nonprofits: Nava, the developer of the chatbot, and Imagine LA, a non-profit supporting families navigating benefit programs. Each participant was presented with a series of realistic, policy-relevant client scenarios, covering common SNAP-related topics such as eligibility rules and documentation requirements. Participants were randomly assigned to one of two conditions. In the treatment group, navigators viewed AI-generated suggestions from Nava’s chatbot before submitting their answers. In the control group, participants completed the same set of questions without chatbot assistance. To simulate real-world imperfections in AI systems, we also experimentally varied the accuracy of chatbot responses across participants, which allows us to examine how variation in suggestion quality influences human judgment.

To evaluate the chatbot’s overall effectiveness, we collected both objective and subjective measures: (1) accuracy of responses benchmarked against expert-coded policy answers, (2) time spent per question, (3) self-reported confidence in responses, and (4) perceived administrative burden. We also tracked variation in performance by question type and client profile to assess whether AI support helped reduce disparities in navigator performance.

This research provides insights into how generative AI can support public-serving roles by improving accuracy, consistency, and efficiency in benefit navigation. It contributes to scholarship on human-AI interaction in policy contexts and highlights how AI tools might reduce or exacerbate disparities in service delivery. Findings inform the responsible design of civic technology and broader debates on AI accountability, fairness, and the evolving role of automation in government services

Authors

• Zhaowen Guo, Georgetown University
  Presenting Author

• Eric Giannella, Georgetown University
  Non-Presenting Co-Author