Tracking and Identifying International Propaganda and Influence Networks Online

Hans W. A. HanleyAAAI 2025 · DOI

TL;DR

This paper introduces a novel AI system designed to automatically identify, track, and analyze the spread of propaganda and misinformation across thousands of international news websites. By leveraging advanced multilingual embeddings and network inference techniques, the system aims to provide a scalable and real-time solution to combat the growing global threat of disinformation.

What Problem Does This Paper Address?

The proliferation of misinformation and propaganda online poses a significant threat to societal trust, public health, and democratic processes. Traditional methods for investigating these phenomena are often qualitative, time-consuming, and reactive. This paper addresses the critical need for a scalable, automated, and real-time system that can effectively track the origins, spread, and influence of harmful narratives across a vast landscape of international news sources. The challenge lies in processing massive amounts of multilingual text data, identifying subtle biases and factual inaccuracies, and uncovering the complex networks through which disinformation propagates.

Methods

The proposed system employs a multi-stage approach:

  • Narrative Tracking:
    • Customized Matryoshka Embeddings: Instead of standard embeddings, the authors utilize a novel multilingual adaptation of Matryoshka embeddings. These embeddings learn progressively higher levels of detail across their dimensions, allowing for differentiation of news articles at the event, topic, and theme levels. This is crucial for understanding nuanced similarities and differences between articles.
  • Hierarchical Level-wise Clustering: A novel bottom-up agglomerative clustering approach, based on an online version of the Reciprocal Nearest Neighbor algorithm, is used to group similar news stories. This method leverages the hierarchical structure of the Matryoshka embeddings.
  • Multilingual Summarization and Keyword Extraction: To make clusters human-interpretable, an open-source decoder LLM is fine-tuned for multilingual summarization. Pointwise mutual information is then used to extract descriptive keywords.
  • Propaganda and Influence Network Identification:
    • Zero-Shot Multilingual Stance Detection: This technique assesses the bias and factual consistency of news articles by determining their stance towards specific topics or entities. It allows the system to identify websites that consistently spread slanted or false content.
  • Network Inference: Using the NETINF algorithm, the system uncovers underlying relationships between websites based on the temporal patterns of their news article publications. This helps map out influence networks promoting particular narratives.

Datasets

  • CommonCrawl: Used to compile and refine a list of international news websites.
  • SemEval-22-t8 dataset: Utilized for training and evaluating the multilingual Matryoshka embeddings for news article similarity.
  • Miranda et al. (2018) news article clustering dataset: Used for benchmarking the clustering approach.

Key Insights and Ideas

  • Hierarchical Embeddings for Nuanced Similarity: The Matryoshka embeddings offer a more granular understanding of news article similarity than traditional methods, allowing for distinctions at event, topic, and theme levels. This is a significant step beyond simple cosine similarity.
  • Scalable Identification of Propaganda: The combination of stance detection and network inference provides a powerful, automated way to identify websites actively spreading disinformation and to map their influence.
  • Tracking Shifting Narratives: The system's ability to track changing attitudes of websites towards topics over time is crucial for understanding evolving propaganda campaigns.
  • Real-time Global Monitoring: The proposed approach moves beyond reactive analysis to enable real-time tracking of disinformation spread across a vast, multilingual news landscape.

Key Results

  • The proposed Matryoshka embedding approach achieved state-of-the-art results on the SemEval-22-t8 dataset, with a Pearson correlation of p = 0.833 with human labels of article similarity.
  • The clustering approach was benchmarked against a multilingual news article clustering dataset and achieved state-of-the-art results.
  • Qualitative examples include identifying that prominent stories in 2022-2023 on English-language outlets concerned inflation and the Russo-Ukrainian War.
  • The system observed a polarization of attitudes toward artificial intelligence in online newspapers starting in December 2022, with hollywoodreporter.com being notably negative due to concerns about AI's impact on writers and actors.
  • Previous work using this methodology identified pandemic.news and childrenshealthdefense.org as highly influential anti-vaccine websites.

Limitations and Future Work

  • Multilingual Keyword Extraction: Current keyword extraction methods (pointwise mutual information) are monolingual. The authors are exploring different avenues for multilingual keyword extraction using decoder LLMs.
  • Expanding Language Support: While the Matryoshka embeddings are trained on 54 languages, the current stance detection model is primarily English. Future work includes developing a multilingual stance detection model with high accuracy across these languages.
  • Understanding Cross-Country Information Spread: The authors plan to apply the methodology to understand how valenced information spreads between and within individual countries.
  • Public Deployment: A significant future goal is to publicly deploy the system to assist journalists and researchers in understanding the spread of misinformation.

Summary updated

BibTeX
@inproceedings{hanley2025tracking,
  author = {Hans W. A. Hanley},
  title = {{Tracking and Identifying International Propaganda and Influence Networks Online}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35209},
}
Figure