Citizenship in a Self-Checkout State: Heteromation vs Automation in Government

• In Event: Algorithms and Democracy

Abstract

Emerging digital technologies and artificial intelligence (AI) systems are rampantly colonizing the human lifeworld including state institutions. This trend tasks political theorists with anticipating how these emerging technologies will transform those state institutions and the accompanying normative consequences. Recent literature has responded to this challenge by developing theories of algocracy as a new regime type defined by the increasing application of AI systems and digital technologies to state functions. Current discussions of algocracy center on a “self-driving car” model in which the application of AI systems in state institutions, as with vehicles, leads to the gradual automation and replacement of human labor by new technologies.

I argue that what has been neglected in this discussion is the potential for algocracy to follow an alternative “self-checkout machine” model in which emerging technologies lead to heteromation effects. Heteromation occurs when technologies rearrange who performs which tasks in society, such as self-check machines that extract unpaid labor from customers, rather than replacing labor through automation. Applied to state institutions under algocracy, technologies for heteromation can similarly extract unpaid labor and desired patterns of behavior from citizens. I contend that the “self-checkout” model of algocracy raises an important and neglected set of normative questions. For instance, a “self-checkout” model of algocracy points us to issues like labor exploitation and digitized administrative burdens. Critically within this paper, an important question to consider is how do we reconceptualize the nature of citizen engagement within local communities when those behaviors are induced/extracted by heteromating technologies?

This paper adopts an interdisciplinary approach that brings critical algorithmic studies into conversation with political theory. I offer three contributions to the emerging discussion regarding algocracy. First, I develop a “self-checkout” model of algocracy. Second, I argue that, given the technical limitations of modern second-wave AI systems, for the foreseeable future algocracy will often follow the “self-checkout” model over the “self-driving car” model. Finally, I offer a normative critique concerning the nature of citizenship in a “self-checkout” algocracy in which AI systems extract civic engagement from citizens. I contend specifically that “self-check” algocracy produces a hollow imitation of actual active citizenship.

Author

• Matthew Lucky, University of Chicago