TL;DR
This paper presents a system for detecting persuasion techniques in text and images, framing it as a multi-label classification problem. The authors leverage powerful pre-trained language models like RoBERTa for text analysis and VGG-16 for image features, combining them in a multimodal approach for a more comprehensive understanding of persuasive content.
What Problem Does This Paper Address?
In today's digital landscape, disinformation campaigns are a significant concern, often employing sophisticated persuasion techniques to influence public opinion. SemEval-2021 Task 6 specifically targets the automatic detection of these techniques, recognizing that how a message is communicated is as crucial as its content. The task is divided into subtasks focusing on text-only analysis (identifying techniques and their spans) and a multimodal approach that considers both text and images within a meme. This research aims to build robust models capable of identifying these persuasive strategies, contributing to the broader field of fake news and disinformation detection.
Methods
The core of the authors' approach relies on RoBERTa, a robustly optimized BERT pre-training approach, for text analysis. This powerful language model is fine-tuned on the provided annotated data for classification tasks. For the multimodal subtask, they incorporate image features by using a pre-trained VGG-16 model.
Key methodological components include:
- Textual Analysis (Subtask 1 & 2): RoBERTa is used for both multi-label classification (Subtask 1) and sequence tagging (Subtask 2). For Subtask 2, a Conditional Random Field (CRF) layer is added on top of RoBERTa to model dependencies between predicted labels, improving sequence tagging accuracy.
- Multimodal Analysis (Subtask 3): This involves a cross-modality model built using attention layers, inspired by Transformer architectures. The model processes image features from VGG-16 and text embeddings from RoBERTa. It employs self-attention within each modality and cross-attention layers to fuse information between text and image representations, enabling a joint understanding of persuasive techniques.
- Input Embeddings: For text, a WordPiece tokenizer is used, and words are projected into vectors, combined with positional embeddings. For images, VGG-16 extracts feature vectors.
Datasets
The dataset used for this task is provided by Dimitrov et al. (2021). The paper also references the "SemEval 2020 task 11 on Detecting propaganda techniques in news articles" as a precursor and basis for Subtask 2.
Key Insights and Ideas
- The Power of Pre-trained Models: The paper highlights the effectiveness of fine-tuning large pre-trained language models like RoBERTa for NLP tasks, achieving competitive results even with limited data.
- Bridging Text and Image: The multimodal approach demonstrates the importance of considering both visual and textual elements in memes to accurately detect persuasion techniques. The cross-attention mechanism is key to learning these inter-modal relationships.
- CRF for Sequence Tagging: The integration of a CRF layer with RoBERTa for sequence tagging (Subtask 2) effectively models label dependencies, leading to improved performance compared to token-level classification alone.
- Data Scarcity as a Challenge: The authors acknowledge that data scarcity is a significant hurdle, particularly in the multimodal setting, impacting the potential for even better results.
Key Results
The authors participated in Subtasks 1 and 3. Their system achieved 9th place out of 16 teams for Subtask 1 and 13th place out of 15 teams for Subtask 3 in the official evaluation. While not top-ranked, their results are presented as strong baseline models that can be further improved.
Limitations and Future Work
- Data Scarcity: The primary limitation identified is the scarcity of training data, especially for the multimodal task, which hinders achieving optimal performance.
- Future Directions: The authors plan to investigate more recent neural architectures like T5 and GPT-3 for language representation. For multimodal settings, they suggest exploring alternative image representation models such as ResNet. They also express interest in applying concepts from active learning, semi-supervised learning, and weak supervision to address data scarcity challenges. Additionally, they see opportunities for transfer learning from related fields like argumentation mining and offensive language detection.
Summary updated