Prta: A System to Support the Analysis of Propaganda Techniques in the News

Giovanni Da San Martino, Shaden Shaar, Yifan Zhang, Seunghak Yu, Alberto Barrón-Cedeño, Preslav Nakov

Introduction

The fight against disinformation has traditionally focused on veracity—determining whether a news source is "fake" or "real." However, propaganda often transcends simple factual accuracy, employing sophisticated rhetorical and psychological techniques to manipulate reader perception. Prta (Propaganda Persuasion Techniques Analyzer), introduced at ACL 2020, shifts the focus from binary classification to the identification of the specific mechanisms of delivery used in manipulative text.

Methodology: Multi-Granularity Gating

Prta utilizes a supervised multi-task learning framework built upon a BERT backbone. The system is designed to handle two distinct but related tasks: Sentence-Level Classification (SLC), which determines if a sentence contains propaganda, and Fragment-Level Classification (FLC), which identifies the exact spans of text corresponding to specific techniques.

A novel aspect of the architecture is the gating mechanism. As shown in the model diagram, the output of the sentence-level classifier acts as a "gate" ($f$) for fragment-level predictions. If the model determines a sentence is unlikely to contain propaganda, it suppresses potential fragment-level detections within that sentence, significantly reducing false positive rates.

The architecture of the multi-granularity gated BERT model, illustrating how sentence-level predictions gate token-level labels.

Data and Propaganda Techniques

The system is trained on a corpus of 350,000 tokens manually annotated for 18 distinct propaganda techniques. These range from logical fallacies like "Whataboutism" and "Straw Man" to emotional appeals such as "Loaded Language" and "Flag-Waving."

Analysis of the dataset reveals a heavy skew toward certain techniques. "Loaded Language" is by far the most frequent, appearing in nearly half of the annotated instances, followed by "Appeal to fear-prejudice" and "Name Calling/Labeling."

Distribution of the 18 propaganda techniques identified in the news corpus, showing the dominance of Loaded Language.

Key Results and Insights

The multi-task gated model demonstrates superior performance over standard BERT baselines. It achieved an F1 score of 60.71 for sentence-level detection and 22.58 for fragment-level detection. While the fragment-level F1 is numerically lower, it represents the state-of-the-art for this challenging task, outperforming previous benchmarks from dedicated NLP hackathons.

Beyond raw performance, Prta enables the analysis of "propaganda signatures." By aggregating data across outlets, the researchers found that different media organizations favor different manipulative styles. For instance, while both the BBC and Fox News utilize loaded language, Fox News showed a higher propensity for "flag waving" and "slogans" in specific contexts.

Limitations and Future Directions

Currently, Prta is optimized for English-language news and struggles with techniques that require inter-article context, such as propaganda by omission or cherry-picking. Future iterations aim to expand language support and develop methods for detecting broader contextual manipulation. By highlighting exactly how a text attempts to persuade, Prta serves as a critical tool for fostering media literacy in an era of weaponized information.

Summary updated

BibTeX
@inproceedings{martino2020prta,
  author = {Giovanni Da San Martino and Shaden Shaar and Yifan Zhang and Seunghak Yu and Alberto Barrón-Cedeño and Preslav Nakov},
  title = {{Prta: A System to Support the Analysis of Propaganda Techniques in the News}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-demos.32},
}
Figure