TL;DR
This paper proposes an interpretable framework to analyze linguistic variations in Russian state media and social media during the Russo-Ukrainian War. By employing methods like Kullback-Leibler Divergence (KLD) and surprisal, the authors aim to uncover how propaganda strategies evolve and differ across these platforms, offering a more transparent approach than traditional black-box NLP models.
What Problem Does This Paper Address?
The full-scale Russian invasion of Ukraine in February 2022 brought a surge in pro-Kremlin propaganda, disseminated through both official state media and social media platforms. Understanding how this propaganda is crafted and evolves is crucial for combating disinformation. While existing NLP methods can detect propaganda, they often lack transparency, making it difficult to understand why a text is classified as propaganda. This paper addresses this gap by focusing on linguistic variation and change as key indicators of propaganda strategies, aiming to provide interpretable insights into how language is manipulated to serve ideological goals. The authors specifically investigate how language use differs between state media and social media, and how these patterns change over time, particularly in the context of the war.
Methods
The paper proposes a research framework that leverages interpretable methods for analyzing linguistic variation and change:
- Kullback-Leibler Divergence (KLD): This statistical measure quantifies the divergence between two probability distributions. Here, it's used to compare the distribution of linguistic features (e.g., words, part-of-speech tags) between state media and social media, and across different time periods. KLD helps identify features that are disproportionately emphasized in one corpus over another, offering a nuanced view beyond simple frequency counts.
- Surprisal: Based on information theory, surprisal measures the unexpectedness of a word given its preceding context. The authors hypothesize that terms used in propaganda, like "special military operation," will initially have high surprisal but become more conventionalized (lower surprisal) over time as they are repeatedly used.
- Word Embeddings: These represent words as vectors in a semantic space, allowing for the analysis of semantic shifts. The authors suggest visualizing how words related to propaganda narratives (e.g., "Nazi Ukraine") change their proximity to other words over time, indicating shifts in their contextual meaning and propaganda framing.
These methods are chosen for their interpretability, aiming to provide a deeper understanding of how propaganda techniques are linguistically constructed and evolve, in contrast to more opaque deep learning models.
Datasets
The primary dataset used is the Wartime Media Monitor (WarMM-2022) corpus. This corpus, created by Alyukov et al. (2023), contains approximately 1.7 million posts related to the Russo-Ukrainian War. It is divided into two parts:
- State-controlled mass media: Includes press and TV transcripts, characterized by a more formal style.
- Social media posts: A larger collection with a less formal register, encompassing content from regime supporters and anti-government voices.
The state media subcorpus covers February to September 2022, while social media posts span July to September 2022, allowing for diachronic analysis.
Key Insights and Ideas
- Interpretability is Key: The paper strongly advocates for interpretable methods (KLD, surprisal) over black-box NLP models for propaganda detection, emphasizing the need to understand why a text is classified as propaganda.
- Media Type Matters: State media and social media likely employ different propaganda frames and strategies due to targeting different audiences (passive vs. active information seekers).
- Language Evolves with Propaganda: Propaganda narratives are not static; they adapt to changing circumstances and aim to conventionalize specific terms and frames over time.
- Euphemisms as a Strategy: State media might favor euphemisms (e.g., "special military operation" instead of "war") to downplay the severity of events, a strategy that could be detected through linguistic analysis.
- Disinformation on Social Media: Social media might be used to spread disinformation and discourage users from seeking alternative news sources, potentially indicated by the prevalence of terms like "truth" and "fact" in a specific context.
Key Results
The paper presents preliminary findings from a pilot study using a small subset of the WarMM-2022 corpus (July 30-31, 2022, comparing nouns):
- State Media: Primarily uses euphemisms like "special military operation," "situation," and "demilitarization."
- Social Media: More frequently uses the direct term "war" (война) and also shows a high contribution of words like "truth" (правда) and "fact" (факт).
These findings align with previous studies showing distinctions between state-affiliated and independent outlets and suggest the use of disinformation frames on social media.
Limitations and Future Work
Limitations: * The current study is a pilot, using a small subset of the WarMM-2022 corpus for initial experiments. * Propaganda identification can be inherently biased due to the political stance of the researcher, though the authors aim to mitigate this through data-driven approaches and reliance on related work. * There is no single, universally agreed-upon definition of propaganda or a unified framework for its detection.
Future Work: * Conduct a comprehensive KLD analysis on the entire WarMM-2022 corpus, comparing state and social media posts and studying diachronic linguistic change. * Apply surprisal and word embeddings to further investigate propaganda strategies and their evolution. * Explore graph neural networks for semantic change and disinformation detection. * Combine interpretable methods with transformer-based approaches for a more robust propaganda detection system. * Extend the research to analyze linguistic variations across different languages (Russian, Ukrainian, English). * Investigate pro-Kremlin propaganda that preceded the full-scale invasion of Ukraine (e.g., since 2014).
Summary updated