Power in the Loop: Algorithmic Governance and Legitimacy under Authoritarianism

• In Event: Digital (Un)Democracy: AI, Media, and the New Political Order

Abstract

Automated decision-making (ADM) systems are increasingly embedded in state governance. Existing research, largely grounded in democratic contexts, often assumes that human involvement enhances legitimacy by providing accountability and moral judgment. This article revisits that assumption by examining how human-in-the-loop shapes the legitimacy of algorithmic governance under authoritarianism. We define legitimacy in behavioral terms, focusing on individuals’ willingness to express discontent at the decision.

We study China’s COVID-19 Health Code system, a central instrument of pandemic governance that combined algorithmic risk assessment with discretionary human intervention. Focusing on moments of misclassification, we analyze how respondents evaluate quarantine decisions when they were confident that they were not at risk. We ask whether attributing the final quarantine determination to a human-in-the-loop algorithmic process, rather than to a big-data model alone, alters the perceived legitimacy of the decision. Using mixed-effects models, we find that explicit human involvement significantly reduces perceived legitimacy. Respondents are less likely to characterize a decision as reasonable when human officials are involved than when the same decision is framed as fully automated. This effect persists after controlling for trust in local and central government.

We further examine heterogeneity in relative legitimacy assessments. Individuals with stigmatized ascribed identities, including ethnic minorities, women, and rural hukou holders, are significantly more likely to view fully automated ADM as more legitimate than human-intervened systems. These patterns suggest anticipations of discriminatory or unequal treatment by human officials. By contrast, Communist Party members are more likely to view human involvement as more legitimate, consistent with their privileged position within informal and relational governance networks that benefit from discretion and flexibility.

By shifting attention from developed democracies to authoritarian contexts, this study shows that human oversight does not universally enhance legitimacy. In authoritarian governance, automation can sometimes appear more legitimate than human discretion, particularly for socially disadvantaged groups.

Authors

• Zheng Fu, Stony Brook University

• Chuncheng Liu, Northeastern University