Abstract
Pretrained Transformers (PTs) are widely celebrated for their superior Out-of-Distribution (OOD) robustness compared to traditional architectures. However, their performance in real-world settings involving noisy data remains a critical concern. This study evaluates 10 model architectures—ranging from Bag-of-Words (BOW) and LSTMs to state-of-the-art Transformers—and finds empirical evidence that PTs provide less robust representations than traditional models when exposed to poisoned training data. Furthermore, we demonstrate that adversarial filtering (AF) mechanisms, intended to improve generalization, are easily subverted by label noise.
Introduction
In real-world applications, datasets are rarely pristine. Data poisoning—where labels are incorrect due to annotator error, unclear instructions, or malicious intent—is an inevitable challenge. While recent research has lauded PTs like BERT and RoBERTa for handling distribution shifts, their susceptibility to noisy training data has been under-explored. We investigate whether the massive scale and self-supervised pre-training of Transformers help them ignore poisoned samples or, conversely, make them more prone to overfitting on noise.
Methodology
We conducted a comparative study across 10 architectures using sentiment classification benchmarks (SST-2 and IMDB). To simulate poisoning, we employed a label-flipping method, creating training sets with 0%, 30%, 50%, 70%, and 90% poisoned data.
To quantify sensitivity to noise, we introduced the Mean Rate of Change of Accuracy with Change in Poisoning (MRAP). A higher MRAP indicates a model that is more sensitive (less robust) to noise. We also developed AFPLite (Adversarial Filtering of Poisoned Data), an adaptation of AFLite designed to prune poisoned samples using RoBERTa embeddings before training.
Analysis of Robustness
Our experiments reveal a characteristic "U-shaped" accuracy curve across all models as poisoning levels increase. As noise approaches 50%, accuracy drops significantly as the model encounters maximum label conflict. However, beyond 50%, accuracy begins to increase again, indicating that the models effectively learn the "flipped" logic of the poisoned labels rather than ignoring them.
Crucially, we find that accuracy on clean data is a deceptive metric for robustness. Models that achieve the highest accuracy on clean benchmarks (Transformers) are often the most sensitive to noise. Simpler models like LSTMs and Bag-of-Words (BOW) exhibited higher stability across varying noise levels.
Key Results
The comparison between clean accuracy and the Normalized MRAP (NMRAP) highlights a significant trade-off. While Transformers dominate on clean datasets, they exhibit low robustness scores. Quantitatively, traditional models like LSTMs achieved a perfect 1.0 on the NMRAP scale (indicating maximum robustness), while Transformers scored as low as 0.14.
This suggests that the quantity and diversity of pre-training data do not inherently protect a model from being "fooled" by noisy fine-tuning data. Furthermore, our AFPLite mechanism failed to reliably distinguish between poisoned and non-poisoned data, suggesting that the "shortcuts" Transformers find in data are present in both clean and noisy samples.
Conclusion
This study provides empirical evidence that Pretrained Transformers are less robust to noisy data than traditional models. Robustness to label noise is a distinct property from OOD generalization, and high performance on clean benchmarks does not guarantee resilience in the real world. Future research should focus on developing training objectives specifically designed to ignore poisoned samples rather than relying solely on the scale of pre-training.
Summary updated