What Does The Public Say? A Computational Analysis of Regulatory Comments

• In Event: Text as Data: Topic Models and Conversations

Abstract

Administrative agencies in the United States, such as the FCC, EPA, and SEC, have the responsibility of implementing laws through the promulgation of regulations. In recent years, as both the ease of participation and interest in rulemaking have grown, there has been an explosion of public participation, and agencies now receive millions of comments each year concerning proposed agency actions. Regulatory agencies are then required to review and respond to these comments, in a painstakingly manual manner, as part of the notice-and-comment process. As these comments are submitted by a wide range of stakeholders, including affected companies, advocacy groups, and the general public, they represent a diversity of perspectives and arguments in support and opposition of the proposals.

In this study, we perform an analysis of over 12 million publicly released comments across agencies in order to examine the ways in which rule and agency characteristics affect the submitted comments. Specifically, we explore how number of comments, the proportion of public, business, and advocacy group authors, and comment contents vary based on the individual rule and agency, and between independent and executive agencies.

Furthermore, we employ computational text analysis to automatically analyze the comment text for stylistic quality (length and complexity), and thematic content. We propose a set of thematic aspects contained in comments, such as the rule being burdensome, lacking clarity, or overreaching, and annotate comments at the sentence level using a rule-based model with one or more themes. We then construct a machine learning model to predict the themes in each comment. Using the predicted themes, we are able to examine how the expert commenters - businesses and advocacy groups - differ from the general public, and how comments submitted to different agencies differ in what they say and how they say it. We use this analysis to compare the commenting characteristics for the first 2 years under the Obama administration and the Trump administration.

Authors

• Vlad Eidelman, FiscalNote

• Michael A. Livermore, University of Virginia