Robustness of Misinformation Classification Systems to Adversarial Examples Through BeamAttack

Arnisa Fazla, Lucas Krauter, David Guzman Piedrahita, Andrianos Michail

Modern misinformation detection systems, while sophisticated, often possess a "glass jaw" when faced with intentional, small-scale perturbations. Researchers from the University of Zurich and Amsterdam UMC recently presented a study at the CLEF 2024 CheckThat! Lab exploring these vulnerabilities. By extending the BeamAttack algorithm, they demonstrated that even state-of-the-art classifiers like BERT and RoBERTa can be systematically deceived into misidentifying fake news or propaganda.

The Adversarial Challenge

Social media platforms increasingly rely on machine learning to filter harmful content. However, adversarial attacks—intentional modifications to input text—can cause a model to flip its prediction, such as labeling "fake news" as "credible." Most existing attacks rely on "greedy search" methods, which change words sequentially until a model fails. These often result in sub-optimal, bulky changes. This research addresses the need for "grey-box" evaluation tools capable of finding the minimal changes required to deceive a model while maintaining human readability.

Methodology: Enhancing BeamAttack

The authors built upon BeamAttack, a search-based attack that explores multiple potential modifications simultaneously (a "beam") rather than a single immediate change.

Overview of the BeamAttack process, illustrating word masking, substitution generation using RoBERTa, and beam search optimization.

Key innovations introduced in this work include:

  • Word Deletion: Unlike previous iterations that only substituted words, this version can remove words entirely to trigger misclassification.
  • LIME Integration: The team integrated Local Interpretable Model-agnostic Explanations (LIME) to identify tokens that most heavily influence a model's decision, rather than relying solely on raw model outputs.
  • Contextual Substitutions: A Masked Language Model (RoBERTa) was used to suggest replacement words that fit the grammatical context.
  • Beam Search Optimization: Hyperparameters like beam size ($k$) and branching factor ($b$) were tuned to balance attack success against the computational cost of querying the victim model.

Experimental Setup

The study utilized the BODEGA framework, covering five distinct misinformation detection tasks: News Bias (HN), Propaganda Identification (PR2), Fact Checking (FC), Rumor Detection (RD), and COVID-19 Misinformation (C19).

Key Results and Insights

The extended BeamAttack achieved a near-perfect success rate, frequently outperforming established baselines like BERT-ATTACK and DeepWordBug across various models and datasets.

Comparative performance of BeamAttack against other adversarial methods across various models and misinformation detection tasks.

The Robustness Hierarchy

Simpler models like BiLSTM were the most vulnerable. While BERT showed more resilience, adversarially trained RoBERTa proved the toughest to crack, requiring larger beams and significantly more queries to find successful adversarial samples.

The Semantic Gap

A critical finding was the tension between "fooling the machine" and "fooling the human." While the automated attack success rate exceeded 99%, human evaluation revealed a major semantic gap. Only 7% of adversarial samples preserved the original meaning, and 30% were deemed nonsensical, often due to the new word-deletion feature breaking sentence structure.

Syntactic Stability

In approximately 44% of successful single-word attacks, the Part-of-Speech (POS) tag of the word remained unchanged. The attack often swaps a noun for a noun or a verb for a verb, maintaining the sentence's "shape" even as the meaning drifts.

Limitations and Future Directions

The research identified several failure modes for BeamAttack:

  1. Grammar Breakdowns: Word removal often leads to syntactically broken sentences that are easily flagged by humans.
  2. Semantic Inversion: The Masked Language Model frequently suggests antonyms, which tricks the classifier but fundamentally changes the text's meaning (e.g., turning a "lie" into a "truth").
  3. Computational Cost: The reliance on LIME and large beam widths makes the attack resource-intensive and difficult to scale.

Future work will focus on using Large Language Models (LLMs) to generate more fluent substitutions and implementing stricter similarity filters to ensure adversarial text remains stealthy and meaningful to human readers.

Summary updated

Abstract

We extend BeamAttack, an adversarial attack algorithm designed to evaluate the robustness of text classification systems through word-level modifications guided by beam search. Our extensions include support for word deletions and the option to skip substitutions, enabling the discovery of minimal modifications that alter model predictions. We also integrate LIME to better prioritize word replacements. Evaluated across multiple datasets and victim models (BiLSTM, BERT, and adversarially trained RoBERTa) within the BODEGA framework, our approach achieves over a 99\% attack success rate while preserving the semantic and lexical similarity of the original texts. Through both quantitative and qualitative analysis, we highlight BeamAttack's effectiveness and its limitations. Our implementation is available at https://github.com/LucK1Y/BeamAttack
BibTeX
@inproceedings{fazla2025robustness,
  author = {Arnisa Fazla and Lucas Krauter and David Guzman Piedrahita and Andrianos Michail},
  title = {{Robustness of Misinformation Classification Systems to Adversarial Examples Through BeamAttack}},
  booktitle = {CLEF},
  year = {2025},
  doi = {10.1007/978-3-032-04354-2\_7},
}
Figure