Abstract
Modern NLP models often achieve high benchmark scores by exploiting spurious correlations rather than true linguistic understanding, leading to failures on out-of-distribution data. This survey unifies the fragmented landscape of robustness research, offering a conceptual schema to define, identify, and mitigate these failures across both adversarial and natural distribution shifts.
Introduction: The Problem of Brittleness
Modern NLP models, particularly large pre-trained language models, achieve incredible scores on standard benchmarks. However, they are often "right for the wrong reasons." They frequently rely on spurious correlations—statistical shortcuts in the training data (like specific keywords or sentence structures) that correlate with labels but don't represent true linguistic understanding.
When these models encounter "out-of-distribution" data—such as a typo, a different dialect, or a strategically placed distracting sentence—their performance plummets. This lack of robustness hinders the safe deployment of AI in critical areas like healthcare or legal tech. The authors tackle the fragmented nature of robustness research, which has historically treated "adversarial attacks" (maliciously crafted inputs) and "distribution shifts" (natural variations) as separate problems.
A Unified Taxonomy for Robustness
As a survey paper, the authors propose a conceptual schema to categorize existing work into three main pillars:
- Defining Robustness: Robustness is unified as the model's performance on a test distribution $D'$ that differs from the training distribution $D$. This includes synthetic perturbations (adversarial) and naturally occurring shifts.
- Identifying Failures: This involves human-driven error analysis and "stress tests," as well as model-based "white-box" attacks to find gradients that flip predictions.
- Mitigation Strategies: Strategies range from data-driven techniques like data augmentation to model-based approaches like Group Distributional Robust Optimization (Group DRO) and causal intervention.
Robustness Benchmarks Across Tasks
The paper synthesizes results across several landmark robustness benchmarks. These benchmarks are essential for evaluating how models handle specific types of shifts, from syntactic heuristics to gender stereotypes.
Key benchmarks include:
- Natural Language Inference (NLI): HANS (syntactic heuristics), SNLI-Hard, and ANLI (adversarial NLI).
- Question Answering: AdvSQuAD (distracting sentences) and Natural-Perturbed-QA.
- Paraphrase Identification: PAWS (word scrambling).
- Fairness/Bias: WinoGender and WinoBias (gender stereotypes in coreference resolution).
- Named Entity Recognition: OntoRock and SeqAttack.
Key Insights and Ideas
- The Root of Evil (Spurious Correlations): Most robustness failures stem from models learning "shortcuts." For example, an NLI model might learn that the word "not" always implies a contradiction, failing when "not" is used in a neutral context.
- Vision vs. NLP: Unlike computer vision, where adversarial noise is often invisible to humans, NLP perturbations (like swapping a word for a synonym) are perceptible but should be "meaning-preserving." Furthermore, the discrete nature of text makes gradient-based attacks much harder than in the continuous space of images.
- Pre-training is a Double-Edged Sword: While massive pre-training (like GPT or BERT) significantly improves general robustness by exposing models to diverse data, it can also bake in societal biases and "hallucinations" that are hard to prune.
- The Accuracy-Robustness Trade-off: There is often a tension where making a model more robust slightly decreases its accuracy on the original "clean" benchmark, suggesting that benchmarks themselves might be rewarding the use of shortcuts.
Key Results
While the paper is a survey, it highlights several critical findings from the field:
- Adversarial Training Limitations: Training on synthetic adversarial examples doesn't always translate to better performance on natural distribution shifts.
- The Power of Diversity: Increasing the diversity and size of pre-training data is currently one of the most effective ways to improve "out-of-the-box" robustness.
- Metric Deficiency: For text generation (summarization, translation), standard metrics like BLEU or ROUGE are poor indicators of robustness. A model can have a high ROUGE score while completely hallucinating facts.
Future Directions
The authors identify several "open questions" for the community:
- Proactive Discovery: We need ways to automatically discover "unrobust" regions of the input space before deployment.
- Interpretability: Better tools are needed to explain why a model relied on a spurious correlation so we can fix the underlying logic.
- Unified Evaluation: The field lacks a single, standardized "robustness score." The authors call for more unified benchmarking efforts (like CheckList or Robustness Gym) to allow for fair comparisons across different models and tasks.
- Human-like Generalization: Ultimately, the goal is to move toward "human-like" linguistic generalization, where models process information with the same deliberative reasoning humans use to avoid falling for simple tricks.
Summary updated