Abstract
This paper summarizes the findings of the WANLP 2022 Shared Task, the first competition dedicated to detecting fine-grained propaganda techniques in Arabic social media. The task comprised two challenges: multi-label classification of tweets and the detection of specific text spans associated with 20 distinct rhetorical techniques. Using a newly curated dataset of 930 tweets, participants demonstrated that while Arabic-specific Transformer models like AraBERT excel at classification, identifying the exact boundaries of propagandistic language remains a complex problem.
Introduction
Computational propaganda—the use of rhetorical and psychological devices to manipulate public opinion—has become a pervasive issue on social media. While significant research has focused on English-language propaganda, Arabic remains under-addressed despite its global importance and the complexity of its linguistic structure. The WANLP 2022 Shared Task was established to bridge this gap, providing a benchmark for identifying 20 specific propaganda techniques, such as "loaded language," "fear-mongering," and "whataboutism," in Arabic tweets.
Methodology and Task Design
The shared task was organized into two primary subtasks to evaluate different levels of detection granularity:
- Subtask 1 (Multi-label Classification): Participants were required to identify which of the 20 predefined propaganda techniques were present in a given tweet.
- Subtask 2 (Span Detection): This more granular task required identifying both the technique and the specific character-level spans within the text where the technique was applied.
Most participating teams employed Transformer-based architectures. Models pre-trained specifically on Arabic corpora, such as AraBERT, MARBERT, and ARBERT, were the most effective. Advanced strategies included data augmentation via translation of English datasets, ensemble modeling, and the use of Conditional Random Fields (CRF) for sequence labeling in Subtask 2.
Dataset Description
The researchers developed a novel dataset consisting of 930 tweets sampled from 20 major Arabic news outlets (e.g., Al Arabiya, Al Jazeera). The annotation process involved three independent annotators per tweet, followed by a consolidation phase to establish a "gold standard."
The taxonomy of 20 techniques covers a broad spectrum of rhetorical devices. However, the distribution of these techniques in the dataset was highly skewed. "Loaded Language" and "Name Calling/Labeling" were the most frequent, while more complex logical fallacies like "Straw Man" or "Reductio ad Hitlerum" were absent, reflecting the nature of short-form social media communication.
Results and Discussion
The competition saw participation from 14 teams in Subtask 1 and 3 teams in Subtask 2.
- Subtask 1: The top-performing system (NGU_CNLP) achieved a Micro-F1 score of 0.649 using an ensemble of AraBERT models and translation-based data augmentation.
- Subtask 2: The winning team (Pythoneers) reached a Micro-F1 of 0.396, highlighting the significant difficulty of precise span detection compared to tweet-level classification.
A key finding was the superiority of Arabic-specific pre-training. Models like AraBERT consistently outperformed multilingual counterparts like mBERT or XLM-R, suggesting that capturing the nuances of Arabic morphology and dialect is crucial for detecting subtle rhetorical patterns.
Conclusion
The WANLP 2022 Shared Task established a vital baseline for Arabic propaganda detection. While classification performance is promising, the lower scores in span detection indicate that the boundaries of rhetorical devices are often context-dependent and fuzzy. Future work should focus on expanding the dataset to include rare techniques, exploring hierarchical classification, and incorporating multimodal data such as memes where text and imagery interact.
Summary updated