Disinformation Warfare: Understanding State-Sponsored Trolls on Twitter and Their Influence on the Web

Savvas Zannettou, Tristan Caulfield, Emiliano De Cristofaro, Michael Sirivianos, Gianluca Stringhini, Jeremy Blackburn

TL;DR

This paper provides a data-driven autopsy of Russian state-sponsored trolls on Twitter, revealing that while their overall influence on mainstream news virality is minor, they are remarkably effective at "pushing" state-funded media like RT (Russia Today). By analyzing 27,000 tweets through the lens of Hawkes Processes, the researchers uncover a professionalized operation characterized by account "resets," strategic identity shifts, and a preference for desktop-based web clients over mobile apps.

What Problem Does This Paper Address?

While the existence of "troll farms"—specifically Russia’s Internet Research Agency (IRA)—is well-documented by journalists and intelligence agencies, quantifying their actual impact on the digital information ecosystem has remained elusive. Previous research often conflated human trolls with automated bots.

The authors tackle three primary questions:

  1. Modus Operandi: How do these state-sponsored actors behave differently from "normal" users?
  2. Content: What specific narratives and metadata (hashtags, URLs, locations) characterize their output?
  3. Influence: Can we mathematically model how much "credit" these trolls deserve for the spread of information across Twitter, Reddit, and 4chan?

Methods

The researchers employed a multi-faceted analytical approach:

  • Characterization: Standard statistical analysis of account metadata, including creation dates, client types (e.g., Web vs. Mobile), and self-reported locations.
  • NLP & Sentiment: They used Latent Dirichlet Allocation (LDA) for topic discovery and the Pattern library for sentiment and subjectivity analysis.
  • Hawkes Processes: This is the paper's methodological centerpiece. A Hawkes Process is a stochastic model where past events increase the probability of future events (self-exciting). By fitting this model to the appearance of specific URLs across different platforms (Twitter, Reddit, 4chan), the authors could estimate the "strength of connections" between these communities and quantify how much one platform's activity "caused" activity on another.

Datasets

The study relies on several key data sources:

  • Ground Truth Trolls: A list of 2,700 Twitter account IDs identified by the US Congress as linked to the Russian IRA.
  • Troll Tweets: 27,000 tweets from 1,000 of these accounts, extracted from the 1% Twitter Streaming API (Jan 2016 – Sept 2017).
  • Baseline Dataset: 96,000 tweets from 1,000 random Twitter users, matched to have similar posting frequencies to the trolls.
  • Cross-Platform URLs: To measure influence, the authors tracked URLs posted by trolls and compared their appearance on Reddit and the 4chan /pol/ board.

Key Insights and Ideas

  • The "Reset" Strategy: Trolls frequently delete their entire tweet history and change their screen names to adopt new personas. For example, an account might transition from a generic "National American News" profile to a hyper-partisan "Proud Texan" identity just before a major political event.
  • Professionalized Infrastructure: Unlike the average user who tweets from a phone, the majority of Russian trolls used the Twitter Web Client. This suggests a "desk job" environment where operators manage multiple accounts from a browser.
  • Strategic Localization: Trolls often set their locations to "US" or specific German cities to appear local, but their posting times (peaking at 14:00–15:00 UTC) are highly consistent with a 9-to-5 workday in Moscow (UTC+3).
  • The RT Exception: Trolls are not particularly good at making any news go viral. However, when it comes to links from RT (Russia Today), they are significantly more influential than the baseline, acting as a dedicated distribution arm for state-funded narratives.

Key Results

  • Overall Influence: The trolls' influence on the general dissemination of URLs on Twitter was measured at a negligible 0.01%.
  • State-Media Impact: For RT-specific content, the trolls were 4 times more effective at influencing Twitter users and 5 times more effective on Reddit compared to their influence on other news sources.
  • Account Creation Peaks: Significant spikes in account creation occurred roughly one week before the 2016 Republican National Convention and a few days before the 2017 Charlottesville "Unite the Right" rally.
  • Content Length: Trolls tend to post longer, more subjective tweets and are more likely to include exactly one image compared to the baseline.

Limitations and Future Work

The authors acknowledge several constraints:

  • Data Sparsity: Because they relied on the 1% Twitter Streaming API, they likely missed a significant volume of troll activity.
  • Evolving Targets: Shortly after the study, Twitter identified an additional 1,000 troll accounts and 50,000 automated accounts that were not included in this analysis.
  • Platform Silos: The study does not include Facebook data, which is widely considered a primary theater for IRA operations but remains difficult for external researchers to access at scale.
  • Future Directions: The authors suggest that future work should focus on developing more robust, real-time detection tools that can account for the "identity-reset" behavior identified in this paper.

Summary updated

Abstract

Over the past couple of years, anecdotal evidence has emerged linking coordinated campaigns by state-sponsored actors with efforts to manipulate public opinion on the Web, often around major political events, through dedicated accounts, or "trolls." Although they are often involved in spreading disinformation on social media, there is little understanding of how these trolls operate, what type of content they disseminate, and most importantly their influence on the information ecosystem. In this paper, we shed light on these questions by analyzing 27K tweets posted by 1K Twitter users identified as having ties with Russia's Internet Research Agency and thus likely state-sponsored trolls. We compare their behavior to a random set of Twitter users, finding interesting differences in terms of the content they disseminate, the evolution of their account, as well as their general behavior and use of Twitter. Then, using Hawkes Processes, we quantify the influence that trolls had on the dissemination of news on social platforms like Twitter, Reddit, and 4chan. Overall, our findings indicate that Russian trolls managed to stay active for long periods of time and to reach a substantial number of Twitter users with their tweets. When looking at their ability of spreading news content and making it viral, however, we find that their effect on social platforms was minor, with the significant exception of news published by the Russian state-sponsored news outlet RT (Russia Today).
BibTeX
@inproceedings{zannettou2019disinformation,
  author = {Savvas Zannettou and Tristan Caulfield and Emiliano De Cristofaro and Michael Sirivianos and Gianluca Stringhini and Jeremy Blackburn},
  title = {{Disinformation Warfare: Understanding State-Sponsored Trolls on Twitter and Their Influence on the Web}},
  booktitle = {The Web Conference},
  year = {2019},
  doi = {10.1145/3308560.3316495},
}

Loading related content...

Figure