The Generative AI ecosystem of Disinformation, Extremism and FIMI

• In Event: EXTR2 Threat landscape and counter-terrorism1

Abstract

The use of Generative Artificial Intelligence (GenAI) has brought about significant transformations in various sectors, including communication and content creation.
The exploitation of GenAI to foster and boost online hate, extremism and terrorism represents a significant criminological challenge, especially when considering the different dynamics of enagement to radicalisation, visual storytelling, memes warfare, toxic narratives, disinformation, and conspiracy theories.
The emotional communication, amplified by GenAI in unprecedented ways, is a key factor in such a complex post-truth scenario.
Through the creation of content that resonates on a deeply emotional level, GenAI has become a powerful tool of both States, non-State actors and proxy actors, as shown in the ongoing conflicts, seeking to spread extremist ideologies and narratives to attack and weaken democracies, also in terms of Foreign Information Manipulation and Interference (FIMI).
The Malicious Use of GenAI (MUGenAI) allows hostile actors to produce highly engaging and emotionally "weaponising" content which facilitates the radicalisation process by appealing to feelings of anger, fear, anxiety, uncertanity and injustice.
This emotional engagement is critical in drawing individuals into extremist circles, making them more susceptible and vulnerable to extreme ideologies.
Visual storytelling, memes and other ambiguous online contents represent an important extremist resourse to produce information disorder and allow complex and often toxic narratives to be simplified and spread rapidly in the cyber-social ecosystem, in particular among youngsters.
Therefore, MUGenAI make it possible to exploit pre-existing societal tensions and/or fears with the aim to deepen divisions, as well as normalize toxic language, polarize and weaponize communities and individuals, creating fertile ground for both "traditional" extremism as well as the increasing Anti-System & Anti-Government Extremism (ASAGe).

Author

• Arije Antinori, Sapienza University of Rome