Abstract
The InCrediblAE task (Task 6 at CLEF-2024 CheckThat! Lab) investigates the robustness of AI-driven misinformation detection systems against adversarial examples (AEs). By introducing small, meaning-preserving modifications to text, researchers attempted to bypass automated filters across five distinct domains. This paper details the evaluation framework, participant strategies, and the critical trade-off between algorithmic confusion and human-perceived semantic integrity.
Introduction
Deep learning models for text classification often exhibit high accuracy on standard benchmarks but remain "brittle" when faced with adversarial examples. In the context of misinformation, this vulnerability represents a significant security risk. A malicious actor could circumvent automated fact-checkers or propaganda detectors by making minor tweaks to a message that flip a model's prediction while remaining readable to humans. The InCrediblAE task aims to move beyond simple accuracy metrics to evaluate the robustness of these "victim" classifiers.
Methodology
The task employed a dual evaluation process to capture both technical success and human readability.
Automatic Evaluation: The BODEGA Score
The primary metric, the BODEGA score, is a multiplicative index combining three factors:
- Confusion: The success rate in flipping the model's prediction.
- Semantic Similarity: Preservation of original meaning, measured via BLEURT.
- Character Similarity: Visual similarity to the original text, measured via Levenshtein distance.
Manual Evaluation
Human judges reviewed successful attacks to verify if the core message remained unchanged. This step is vital because automated metrics often miss subtle logical shifts—such as changing a date—that invalidate a fact-check while maintaining high algorithmic similarity.
Datasets
The evaluation utilized the BODEGA framework across five misinformation domains, including a newly introduced COVID-19 dataset.
The domains include:
- HN (Hyperpartisan News): Style-based news bias.
- PR (Propaganda): Manipulative technique identification.
- FC (Fact-Checking): Claim verification against Wikipedia.
- RD (Rumour Detection): Twitter thread analysis.
- C19 (COVID-19): Misinformation related to the pandemic.
Results and Discussion
The results reveal a stark contrast between different attack strategies. While word-swapping techniques often led the automatic leaderboards, they frequently failed human semantic preservation tests.
Key Findings
- The Meaning Gap: Teams dominating the BODEGA leaderboard sometimes preserved original meaning in as few as 7% of cases according to human reviewers.
- Homoglyph Effectiveness: Character-level attacks (e.g., team SINAI) proved highly effective. By replacing standard characters with visually identical "homoglyphs," they achieved a 99% human-verified meaning preservation rate while successfully fooling models.
- Domain Sensitivity: Fact-checking (FC) proved to be the most resilient domain. Because "every word matters" in a factual claim, even minor adversarial tweaks often altered the truth value, making it difficult to generate valid AEs.
- Defense Mechanisms: Adversarial training (e.g., the "Surprise" RoBERTa model) increased the effort required for a successful attack, often demanding thousands of queries, though it was not an absolute defense.
Conclusion
The InCrediblAE task highlights that current misinformation detectors are highly vulnerable to sophisticated adversarial attacks, particularly character-level homoglyph substitutions. Future work should focus on developing automatic metrics that better align with human perception and exploring input sanitization techniques to detect these "stealthy" modifications before they reach the classifier.
Summary updated