System-Level Analysis of Commenter Identity and Agency Uptake in Federal Notice and Comment

• In Event: National Policymaking

Abstract

This research investigates federal notice and comment, and specifically, how the criteria of “interested persons” and “substantial comments” from the Administrative Procedure Act influence agency response in this process. While considered a primary channel for public participation in federal rule-making, recent scholarship suggests this hallowed role has been co-opted by organized interests and repeat players. Moreover, most research on notice-and-comment focuses on a few high-profile rules or on single agencies. This study moves past those single cases to analyze the Federal Register as a whole, quantifying the cited inequalities in this process from 2014 to 2024. 

We utilized a computational pipeline, consisting of processing a stratified sample of comments, applying semantic deduplication, and using large language models (LLMs) to classify commenter identities (e.g., individuals, corporations, experts) and track how agencies responded to these specific comments as the regulations went through stages of notice and comment. We followed established techniques from survey sampling and natural language processing, including the use of FAISS for approximate nearest neighbor search at scale and a human-in-the-loop calibration process to validate LLM-based identity labels and response detection.

Our findings reveal extreme concentration – for example, 0.2 percent of documents receive 90 percent of total comment volume. Moreover, while the 2.6-fold drop in longitudinal agency response rates suggests a move toward automated triage, our own filtering – which mimics agency deduplication – shows that responses to original comments have also declined, indicating that agencies are also engaging less with “substantial” input. We identify a two-tier participatory structure where, while everyday citizens appear to be participating the most, and crowd the top 1% of documents by comment volume, the organizations and corporations account for most of the comments in the other 99% of the documents. Lifecycle tracking also shows that response rates from agencies increase as regulations go through higher stages of NPRMs, and that these organizations maintain significantly more sustained engagement through the final stages of rulemaking. However, while comments from everyday citizens appear to be ignored, unique comments are acknowledged in rules 17.3% of the time, nearly matching the 22.6% rate of large organizations.

Again, this discusses the “substantive” quota that remains a part of regulatory influence. In all, this project highlights the evolving challenges of procedural democracy in an era of automated participation and the response of automated triage. This is especially relevant given the increasing gatekeeping power of the Office of Information and Regulatory Affairs (OIRA), the ongoing “science wars” that have eroded internal agency expertise, and the increase in regulatory dark matter. Our work addresses this tension with the privatization of rule-making, while showcasing the importance of unique submissions for public intervention.

Author

• Fuga Takahashi, Stanford University
  Presenting Author