Introduction
Large Language Models (LLMs) like BERT and RoBERTa have set new benchmarks in Natural Language Processing (NLP). However, their internal mechanisms remain largely opaque. This research investigates whether these transformer-based models possess vulnerabilities similar to the "white noise" susceptibility found in computer vision, where random patterns can trigger high-confidence misclassifications. The authors introduce the Evolutionary Fooling Sentences Generator (EFSG), a black-box adversarial attack designed to expose these weaknesses using Darwinian principles.
Methodology
EFSG is a model-agnostic and task-agnostic evolutionary algorithm. It treats the target model's confidence score for a positive classification as a "fitness" metric. The process begins with an initial population of random word sequences sampled from a training vocabulary. Through successive generations, the algorithm applies selection, crossover, and mutation:
- Selection: Sentences with the highest confidence scores are chosen for reproduction.
- Crossover: Features from successful adversarial examples are combined to explore new regions of the input space.
- Mutation: Random words are swapped to maintain diversity.
This iterative process continues until the algorithm generates "fooling sentences"—nonsense strings that the model classifies as valid English or correct paraphrases with near-total certainty. The inclusion of crossover was found to be critical for overcoming performance plateaus, suggesting that models are sensitive to specific combinations of linguistic features.
Experimental Setup
The study evaluates EFSG against BERT (Base and Large) and RoBERTa on two binary classification tasks from the GLUE benchmark:
- CoLA (Corpus of Linguistic Acceptability): Assessing grammatical correctness.
- MRPC (Microsoft Research Paraphrase Corpus): Identifying semantic equivalence between sentence pairs.
The vocabulary for the generator is derived from the training sets of these corpora to ensure the models are familiar with the individual tokens.
Results and Discussion
The experiments demonstrate that LLMs are indeed susceptible to "textual white noise." EFSG successfully generated gibberish strings that models rated as highly grammatical or semantically equivalent. As shown in the evolutionary plots, the number of successful fooling sentences ($M$) increases significantly over generations, particularly for lower confidence thresholds ($T$).
While RoBERTa showed greater inherent robustness than BERT, it remained vulnerable. To address this, the authors implemented an adversarial training defense. By incorporating the generated fooling sentences into the training set with negative labels, they produced a "BERT improved" model. This version exhibited significantly higher robustness against EFSG attacks while maintaining its original performance on standard benchmarks. Furthermore, the research found that vulnerabilities are often shared: if one fine-tuning of BERT is fooled by a specific nonsense string, there is a 40-60% chance that a different fine-tuning of the same model will also be fooled.
Conclusion
The EFSG framework highlights a significant gap between a model's confidence and its actual linguistic competence. By exposing these "white noise" vulnerabilities, the research provides a pathway toward more interpretable and robust NLP systems. Future work will focus on extending EFSG to non-binary tasks and incorporating constraints to generate more human-readable adversarial examples.
Summary updated