Prta: A System to Support the Analysis of Propaganda Techniques in the News

Giovanni Da San Martino, Shaden Shaar, Yifan Zhang, Seunghak Yu, Alberto Barrón-Cedeño, Preslav Nakov

Abstract: Recent events, such as the 2016 US Presidential Campaign, Brexit and the COVID-19 "infodemic", have brought into the spotlight the dangers of online disinformation. There has been a lot of research focusing on fact-checking and disinformation detection. However, little attention has been paid to the specific rhetorical and psychological techniques used to convey propaganda messages. Revealing the use of such techniques can help promote media literacy and critical thinking, and eventually contribute to limiting the impact of "fake news" and disinformation campaigns. Prta (Propaganda Persuasion Techniques Analyzer) allows users to explore the articles crawled on a regular basis by highlighting the spans in which propaganda techniques occur and to compare them on the basis of their use of propaganda techniques. The system further reports statistics about the use of such techniques, overall and over time, or according to filtering criteria specified by the user based on time interval, keywords, and/or political orientation of the media. Moreover, it allows users to analyze any text or URL through a dedicated interface or via an API.

Introduction

Most research into online misinformation focuses on veracity: is a claim true or false? However, the term "fake news" is often unhelpful because propaganda can be orthogonal to truth. A news story can be factually accurate but still use manipulative framing to influence opinion. While fact-checking organizations are increasingly effective at debunking lies, there is a significant gap in tools that help users recognize how they are being persuaded.

Prta (Propaganda Persuasion Techniques Analyzer) addresses this by focusing on the mechanism of disinformation—the specific linguistic "tricks" used to bypass critical thinking and trigger emotional responses. By moving beyond simple detection, Prta acts as a media literacy assistant, identifying 18 different rhetorical and psychological techniques across thousands of news articles in real-time.

System Architecture and Methods

The core of Prta is a supervised, multi-granularity neural network built on top of BERT. The system treats propaganda detection as two simultaneous tasks:

  1. Fragment-Level Classification (FLC): Identifying the exact spans of text where a technique occurs and labeling it (e.g., identifying a specific phrase as "name calling").
  2. Sentence-Level Classification (SLC): Predicting whether a sentence contains at least one propaganda technique.

The Gating Innovation

A key technical contribution of Prta is the use of a "trainable gate" to link these tasks. If the sentence-level classifier is highly confident that a sentence contains no propaganda, it effectively "mutes" the fragment-level predictions for that sentence. This joint learning approach helps the model use broader context to reduce false positives at the word level.

Architecture of the multi-granularity BERT-based model with a trainable gate.

The system architecture includes a crawler that monitors over 250 RSS feeds and Twitter accounts, extracting and deduplicating content before analysis. This allows for a continuous stream of data that reflects the current media landscape.

Data and Annotation

The model was trained on a specialized corpus of 350,000 word tokens from news articles. Unlike many NLP datasets that rely on crowdsourcing, this corpus was manually annotated by professional annotators to ensure high quality. The dataset includes 18 distinct propaganda techniques, such as Red Herring, Straw Man, Appeal to Fear, and Flag-waving. The authors have made both the corpus and the models publicly available to encourage further research in the field.

Analysis and Visualization

Prta provides a comprehensive dashboard for macro-level analysis of media bias. Users can filter articles by time interval, keywords, and the political orientation of the media outlet (left, center, or right).

Prta system interface showing the distribution of propaganda techniques across analyzed articles.

One of the most common techniques identified by the system is "Loaded Language." The visualization tools allow for fascinating comparisons; for instance, the system can show how different outlets prioritize different techniques, such as a preference for "flag waving" and "slogans" in some media versus "labeling" and "doubt" in others.

Results and Performance

The multi-granularity model outperformed standard BERT baselines, achieving an F1 score of 60.71 at the sentence level and 22.58 at the fragment level. While fragment-level detection remains a significant challenge in NLP, the Prta model outperformed the winning solution of a dedicated "hack-the-news" datathon on the same data. The system successfully scales to process a massive volume of news, providing aggregated statistics that help users understand the broader trends in propaganda use.

Limitations and Future Directions

Despite its success, identifying the exact start and end of a propaganda span remains difficult. Furthermore, the current system is limited to English-language media. Future work will focus on expanding to non-English regions and addressing complex techniques like "cherry-picking" or "omission," which require cross-article analysis to detect what information has been left out. By highlighting these manipulative mechanisms, Prta aims to empower readers to engage more critically with the news they consume.

Summary updated

Abstract

Recent events, such as the 2016 US Presidential Campaign, Brexit and the COVID-19 "infodemic", have brought into the spotlight the dangers of online disinformation. There has been a lot of research focusing on fact-checking and disinformation detection. However, little attention has been paid to the specific rhetorical and psychological techniques used to convey propaganda messages. Revealing the use of such techniques can help promote media literacy and critical thinking, and eventually contribute to limiting the impact of "fake news" and disinformation campaigns. Prta (Propaganda Persuasion Techniques Analyzer) allows users to explore the articles crawled on a regular basis by highlighting the spans in which propaganda techniques occur and to compare them on the basis of their use of propaganda techniques. The system further reports statistics about the use of such techniques, overall and over time, or according to filtering criteria specified by the user based on time interval, keywords, and/or political orientation of the media. Moreover, it allows users to analyze any text or URL through a dedicated interface or via an API. The system is available online: https://www.tanbih.org/prta
BibTeX
@inproceedings{martino2020prta,
  author = {Giovanni Da San Martino and Shaden Shaar and Yifan Zhang and Seunghak Yu and Alberto Barrón-Cedeño and Preslav Nakov},
  title = {{Prta: A System to Support the Analysis of Propaganda Techniques in the News}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-demos.32},
}

Loading related content...

Figure