Abstract
Social media platforms offer unprecedented opportunities for connectivity and exchange of ideas; however, they also serve as fertile grounds for the dissemination of disinformation. Over the years, there has been a rise in state-sponsored campaigns aiming to spread disinformation and sway public opinion on sensitive topics through designated accounts, known as troll accounts. Past works on detecting accounts belonging to state-backed operations focus on a single campaign. While campaign-specific detection techniques are easier to build, there is no work done on developing systems that are campaign-agnostic and offer generalized detection of troll accounts unaffected by the biases of the specific campaign they belong to. In this paper, we identify several strategies adopted across different state actors and present a system that leverages them to detect accounts from previously unseen campaigns. We study 19 state-sponsored disinformation campaigns that took place on Twitter, originating from various countries. The strategies include sending automated messages through popular scheduling services, retweeting and sharing selective content and using fake versions of verified applications for pushing content. By translating these traits into a feature set, we build a machine learning-based classifier that can correctly identify up to 94% of accounts from unseen campaigns. Additionally, we run our system in the wild and find more accounts that could potentially belong to state-backed operations. We also present case studies to highlight the similarity between the accounts found by our system and those identified by Twitter.
Introduction
Detecting state-sponsored influence operations on social media is a persistent challenge. Historically, detection systems have been campaign-specific, trained to recognize keywords or behaviors tied to a single event, such as the 2016 US Election. However, these models are often brittle, failing when new actors emerge or tactics shift. State-sponsored "trolls" are increasingly sophisticated, exhibiting human-like behaviors and blending into communities to avoid detection. This research proposes a campaign-agnostic system designed to identify the "universal fingerprints" of state-backed operations regardless of their origin or target.
Methodology: Identifying Universal Fingerprints
The study analyzed 19 state-sponsored campaigns involving nearly 80,000 accounts and 200 million tweets. To build a generalized classifier, the researchers developed a set of 45 features across four categories:
- User Attributes: Metadata such as account age, follower counts, and tweet frequency.
- Temporal Characteristics: 24-hour activity fingerprints.
- Stylometry: Linguistic markers like sentence complexity and the ratio of function words.
- Source Features: Tracking the specific applications used to post (e.g., "Twitter for iPhone" vs. custom scheduling tools).
The researchers employed a "leave-one-campaign-out" training approach to ensure the model could detect accounts from campaigns it had never encountered during training.
Key Behavioral Insights
The analysis revealed several cross-campaign strategies:
- The "Fake App" Trick: State actors frequently use impersonated third-party applications with names that look legitimate but contain subtle errors (e.g.,
"Twitter for Android "with an extra space). - Camouflage via "Advice": Trolls often post verbatim "advice" messages—inspirational quotes or uplifting content—shared across entirely different campaigns to build social proof and appear human.
- Shared Infrastructure: Synchronized spikes in the use of specific custom applications across different state actors suggest shared tactics or service providers.
- Amplification Tactics: High "Retweet Fractions" were strong predictors of troll accounts, as these operations rely heavily on amplification to force narratives into the public eye.
Results and Real-World Impact
The Random Forest classifier achieved a standard cross-validation accuracy of 98.5%. More importantly, when trained on a single campaign (e.g., the 2018 Russian IRA), it could detect up to 94% of accounts in unrelated, unseen campaigns.
In a real-world test, the system analyzed 2,696 accounts using "fake" applications and identified 116 new malicious accounts not yet officially flagged by Twitter. The system maintained a low false-positive rate of approximately 1.75%.
Conclusion and Future Directions
While the system shows great promise, the evolving landscape of social media presents new hurdles. The discontinuation of free API access limits large-scale real-time testing, and the rise of Large Language Models (LLMs) may alter the stylometric fingerprints of future troll campaigns. Future research must adapt to these generative AI threats and the potential shift toward more decentralized, manual posting methods by state actors.
Summary updated