The Imitation Game: How Digital Emulation Enables Online Influence Operations

• In Event: Understanding Misinformation and Conspiracy Theories

Abstract

Disinformation and misinformation have become compelling social problems commanding significant political and public attention. Evidence of the involvement of both state and non-state actors in constructing and communicating distorted and deceptive public communications has driven significant research into the organisation and conduct of digital information manipulation. It is an effort that has engaged government agencies, commercial businesses, civil society and academia, as well as some social media platforms. Interwoven into the apparent growth, prevalence and distribution of digital disinformation, is what we might label a mounting ‘attribution’ problem. Attribution is the term used by many in the open source analysis community to describe how they assess and make a judgement about who, on the basis of the evidence available, can reasonably be held responsible for designing and delivering specific instances of information manipulation. Attribution thus presents a major political issue for governments, law enforcement and democracy as malign actors are learning and copying “best practices”.

In this paper, we study structured influence operations purposely designed to evade attribution and “deplatforming” through “emulation”. Emulation can be defined as copying, reproducing or imitating a feature in an effort to equal or surpass the impacts of the original that is being copied. Emulators can focus upon issues of account identity, messaging content, or digital behaviours and influencing techniques. To develop these ideas the paper is organised around one case study illuminating a particular facet of emulation as a digital behaviour. In particular, it examines how accounts run by the Russian Internet Research Agency, as the archetypal foreign state influence operation, sought to emulate the presentation and behavior of a particular social movement thus seeking to infiltrate and influence an entire demographic of social media users. Empirical data is drawn from a long-term programme of research exploring the causes and consequences of systematic information manipulation.

Developing the notion of “emulation”, allows us to make three principal claims. First, it provides a way of tracing out how and why the social problem posed by disinformation is evolving and adapting to influence and/or divide everyday citizens, as well as the challenge it is posing to democracy and the political landscapes in Western countries. Second, the concept of emulation helps us to draw out the roots of the growing problem of attribution for research, law enforcement and governments. Stated simply, where different actors are explicitly imitating each other, it becomes harder to confidently interpret who is responsible for what, thus blurring of the lines between different types of malign actor and their motives. Finally, this article finds that in many cases these emulators are becoming significant influencers in their own right, accruing their own kind of political power and influence.

Authors

• Diyana Dobreva, Cardiff University

• Martin Innes, Cardiff University Crime and Security Institute

• Andrew Dawson