Sentiment Analysis and Russian Strategic Narratives towards Ukraine, 2005-2020

• In Event: Other Forms of Power: Soft Power, Mythology, and Rhetoric in Foreign Policy

Abstract

This paper shows how sentiment analysis and strategic narratives (SNs) framework can be used to analyze developments in international relations and foreign policy longitudinally. We show that changes in diplomatic rhetoric tend to be linked to important political and military developments. Rhetorical shifts may foreshadow political shifts and help with identifying opportunities to shape foreign policy. We apply this framework to a case of Russian rhetoric towards Ukraine, 2004-2020. In our case study, we quantitatively and qualitatively analyze more than 20,000 statements by the Russian Ministry of Foreign Affairs (MOFA). This data is particularly useful for studying strategic narratives, as it was purposefully written and posted in English as it was designed to signal Russia’s position to the broader world. Methodologically, we show polarity scores generated by the sentiment analysis and implement change point analysis to identify and date shifts in SNs over time. We apply univariate and vector autoregression models, and test polarity through time. We show that it is possible that sentiment can notably change in some instances when SNs largely remain the same, which can signal important political changes on the ground. For example, there is a clear reduction in both negativity and frequency of the MOFA references to Ukraine in the year preceding Russia’s intervention in the country. We also find that, counterintuitively, there may not be a change in sentiment when politics suggest there should be. A longitudinal approach allows analyzing how SNs evolve through time and to show important nuances, continuity and change. Language and tone, cornerstones of politics, debate, and policy, change across time with intentional, and thereby strategic, purpose in an effort to shape and influence perceptions.

The project aims both to elaborate how the sentiment analysis approach can be successfully used in IR analysis as well as to provide an analysis of Russia’s SNs towards Ukraine during these roughly last 15 years. It will start by explaining how sentiment analysis, a relatively underutilized approach to inquiry in IR, can be used to assess how a country is framing a topic and signaling policy continuity or change. At the same time, the strategic narratives approach bridges the fields of communications and political science and looks at rhetoric and intentions of different actors as they describe various situations in IR. Actors develop such narratives to persuade each other and publics both at home and abroad; and successful SNs can become binding and either shape or put constraints on the actions of countries at home and abroad. A handful of studies utilizing Russian rhetoric and actions towards Ukraine through the SN approach tend to focus on how Russia has articulated its position during and after the 2014 crises in Ukraine.

Our approach will differ from previous studies by looking at broader Russian SN towards Ukraine over a 15-year timeframe and by utilizing the sentiment analysis approach as a means to identify key moments of change. Additionally, in difference from previous analyses utilizing SN frameworks, instead of analyzing select statements or speeches by key leaders, we will analyze all relevant statements by the Russian Ministry of Foreign Affairs.

The paper makes important contributions to the study of strategic narratives, and IR more broadly. First, we convincingly introduce sentiment analysis to the study of strategic narratives thereby providing a quantifiable and time-variant measure of a country’s tone regarding key topics, phrases, and language used to create the SNs. Second, we show how the study of language and tone, cornerstones of politics, debate, and policy, change across time with intentional, and thereby strategic, purpose in an effort to shape and influence perception and strategic narratives, even of such highly contested international conflict and affairs as the armed conflict in Ukraine.

Authors

• Juris Pupcenoks, Marist College

• Scott Fisher, New Jersey City University

• Graig R. Klein, Leiden University