Lawmaker Oversight: Municipal Surveillance Regulation as Algorithmic Accountability in Practice

• In Event: 1555 - Section on Science, Knowledge, and Technology Refereed Roundtables (1 hour)
  In Refereed Roundtable 60min: 5. Scaling Up the Debate: SKAT Perspectives on Public Policies

Abstract

Recognition of the disparate impact of algorithmic technologies on marginalized groups has spurred scholarly attention towards the possibility of regulating algorithmic systems. We examine recent efforts in surveillance regulation as a model for policy approaches to making algorithmic systems publicly accountable. Taking the 2017 Seattle Surveillance Ordinance as our primary case study, and surveying efforts across five other cities, we identify strengths in existing regulation; including procedures for describing the capabilities of surveillance technologies, subjecting them to public oversight, and constraining their allowable uses. However, we also find that expert technologists in city government do not perceive the surveillance technologies under review to be algorithmic or machine learning systems, even as several qualify as such. This finding implicates future efforts toward algorithmic regulation, in that automated functions within systems are often invisible to policy makers, even recognized specialists in information technologies. To refer to this phenomenon, we present the “Minsky Fallacy” to describe the definitional disconnect in which users do not recognize the complexity of a computerized system when it performs a seemingly human task; as well as the “Terminator Fallacy,” a way of conceiving artificial intelligence as limited to systems that present complex agency. The blind spots presented by these fallacies neglect of distinct harms associated with machine learning-powered systems. We argue that finer-grained distinctions between types of algorithmic and information systems in the language of law and policy would strengthen future regulatory efforts, rendering underlying algorithmic components more legible to political and community stakeholders.

Authors

• Meg Young, University of Washington

• Mike Katell, University of Washington

• Peter Martin Krafft, University of California-Berkeley