What Sentiment Analysis Tells Us About Russian Strategic Narratives on Ukraine

• In Event: Propaganda and Influence Campaign Operations

Abstract

This paper will utilize the strategic narratives (SNs) framework and sentiment analysis to assess Russian rhetoric towards Ukraine from 2005-2020. The analysis will be based on a new dataset (FOCUSdata Project) housed at New Jersey City University and funded by the US Office of the Director of National Intelligence (this data also served as the basis for a workshop on Diplomatic Communications at the 2020 APSA).

The project aims both to elaborate how the sentiment analysis approach can be successfully used in IR analysis as well as to provide an analysis of Russia’s SNs towards Ukraine during these last 15 years. It will start by explaining how sentiment analysis, a relatively underutilized approach to inquiry in IR, can be used to assess how a country is framing a topic and signaling policy continuity or change. At the same time, the strategic narratives approach bridges the fields of communications and political science and looks at rhetoric and intentions of different actors as they describe various situations in IR. Actors develop such narratives to persuade each other and publics both at home and abroad; and successful SNs can become binding and either shape or put constraints on the actions of countries at home and abroad. A handful of studies utilizing Russian rhetoric and actions towards Ukraine through the SN approach tend to focus on how Russia has articulated its position during and after the 2014 crises in Ukraine.

Our approach will differ from previous studies by looking at broader Russian SN towards Ukraine over a 15-year timeframe and by utilizing the sentiment analysis approach as a means to identify key moments of change. Additionally, in difference from previous analyses utilizing SN frameworks, instead of analyzing select statements or speeches by key leaders, we will analyze all relevant statements by the Russian Ministry of Foreign Affairs.

The key data for this research consists of all relevant statements from among more than 20,000 cumulative statements posted in English on the Russian Ministry of Foreign Affairs (MOFA) home page between May 2004 and July 2020. This data is particularly useful for studying strategic narratives, as it was purposefully written and posted in English as it was designed to signal Russia’s position to the broader world.

Methodologically, first, we use descriptive statistics of the polarity score generated by the sentiment analysis to illustrate trends and shifts in Russia’s SN. For example, we show significant variation in the frequency and timing in MOFA’s statements regarding Crimea and Donbass beginning in October 2013. We also implement change point analysis to identify and date shifts in the SN over time. We apply univariate and vector autoregression models, and test polarity through time. Finally, we identify and analyze continuation and change in key themes in Russian SNs towards Ukraine.

The paper makes important contributions to the study of strategic narratives, and IR more broadly. First, we convincingly introduce sentiment analysis to the study of strategic narratives thereby providing a quantifiable and time-variant measure of a country’s tone regarding key topics, phrases, and language used to create the SNs. Second, we show how the study of language and tone, cornerstones of politics, debate, and policy, change across time with intentional, and thereby strategic, purpose in an effort to shape and influence perception and strategic narratives, even of such highly contested international conflict and affairs as the armed conflict in Ukraine.

Authors

• Juris Pupcenoks, Marist College

• Scott Fisher, New Jersey City University

• Graig R. Klein, Leiden University