Measure and Improve Robustness in NLP Models: A Survey

Xuezhi Wang, Haohan Wang, Diyi Yang

Abstract

Modern NLP models often achieve high benchmark scores by exploiting spurious correlations rather than true linguistic understanding, leading to failures on out-of-distribution data. This survey unifies the fragmented landscape of robustness research, offering a conceptual schema to define, identify, and mitigate these failures across both adversarial and natural distribution shifts.

Introduction: The Problem of Brittleness

Modern NLP models, particularly large pre-trained language models, achieve incredible scores on standard benchmarks. However, they are often "right for the wrong reasons." They frequently rely on spurious correlations—statistical shortcuts in the training data (like specific keywords or sentence structures) that correlate with labels but don't represent true linguistic understanding.

When these models encounter "out-of-distribution" data—such as a typo, a different dialect, or a strategically placed distracting sentence—their performance plummets. This lack of robustness hinders the safe deployment of AI in critical areas like healthcare or legal tech. The authors tackle the fragmented nature of robustness research, which has historically treated "adversarial attacks" (maliciously crafted inputs) and "distribution shifts" (natural variations) as separate problems.

A Unified Taxonomy for Robustness

As a survey paper, the authors propose a conceptual schema to categorize existing work into three main pillars:

  1. Defining Robustness: Robustness is unified as the model's performance on a test distribution $D'$ that differs from the training distribution $D$. This includes synthetic perturbations (adversarial) and naturally occurring shifts.
  2. Identifying Failures: This involves human-driven error analysis and "stress tests," as well as model-based "white-box" attacks to find gradients that flip predictions.
  3. Mitigation Strategies: Strategies range from data-driven techniques like data augmentation to model-based approaches like Group Distributional Robust Optimization (Group DRO) and causal intervention.
Taxonomy of robustness methods categorized by space and perturbation level.

Robustness Benchmarks Across Tasks

The paper synthesizes results across several landmark robustness benchmarks. These benchmarks are essential for evaluating how models handle specific types of shifts, from syntactic heuristics to gender stereotypes.

Summary of robustness benchmarks across various NLP tasks.

Key benchmarks include:

  • Natural Language Inference (NLI): HANS (syntactic heuristics), SNLI-Hard, and ANLI (adversarial NLI).
  • Question Answering: AdvSQuAD (distracting sentences) and Natural-Perturbed-QA.
  • Paraphrase Identification: PAWS (word scrambling).
  • Fairness/Bias: WinoGender and WinoBias (gender stereotypes in coreference resolution).
  • Named Entity Recognition: OntoRock and SeqAttack.

Key Insights and Ideas

  • The Root of Evil (Spurious Correlations): Most robustness failures stem from models learning "shortcuts." For example, an NLI model might learn that the word "not" always implies a contradiction, failing when "not" is used in a neutral context.
  • Vision vs. NLP: Unlike computer vision, where adversarial noise is often invisible to humans, NLP perturbations (like swapping a word for a synonym) are perceptible but should be "meaning-preserving." Furthermore, the discrete nature of text makes gradient-based attacks much harder than in the continuous space of images.
  • Pre-training is a Double-Edged Sword: While massive pre-training (like GPT or BERT) significantly improves general robustness by exposing models to diverse data, it can also bake in societal biases and "hallucinations" that are hard to prune.
  • The Accuracy-Robustness Trade-off: There is often a tension where making a model more robust slightly decreases its accuracy on the original "clean" benchmark, suggesting that benchmarks themselves might be rewarding the use of shortcuts.

Key Results

While the paper is a survey, it highlights several critical findings from the field:

  • Adversarial Training Limitations: Training on synthetic adversarial examples doesn't always translate to better performance on natural distribution shifts.
  • The Power of Diversity: Increasing the diversity and size of pre-training data is currently one of the most effective ways to improve "out-of-the-box" robustness.
  • Metric Deficiency: For text generation (summarization, translation), standard metrics like BLEU or ROUGE are poor indicators of robustness. A model can have a high ROUGE score while completely hallucinating facts.

Future Directions

The authors identify several "open questions" for the community:

  1. Proactive Discovery: We need ways to automatically discover "unrobust" regions of the input space before deployment.
  2. Interpretability: Better tools are needed to explain why a model relied on a spurious correlation so we can fix the underlying logic.
  3. Unified Evaluation: The field lacks a single, standardized "robustness score." The authors call for more unified benchmarking efforts (like CheckList or Robustness Gym) to allow for fair comparisons across different models and tasks.
  4. Human-like Generalization: Ultimately, the goal is to move toward "human-like" linguistic generalization, where models process information with the same deliberative reasoning humans use to avoid falling for simple tricks.

Summary updated

Abstract

As NLP models achieved state-of-the-art performances over benchmarks and gained wide applications, it has been increasingly important to ensure the safe deployment of these models in the real world, e.g., making sure the models are robust against unseen or challenging scenarios. Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research. In this paper, we aim to provide a unifying survey of how to define, measure and improve robustness in NLP. We first connect multiple definitions of robustness, then unify various lines of work on identifying robustness failures and evaluating models' robustness. Correspondingly, we present mitigation strategies that are data-driven, model-driven, and inductive-prior-based, with a more systematic view of how to effectively improve robustness in NLP models. Finally, we conclude by outlining open challenges and future directions to motivate further research in this area.
BibTeX
@inproceedings{wang2022measure,
  author = {Xuezhi Wang and Haohan Wang and Diyi Yang},
  title = {{Measure and Improve Robustness in NLP Models: A Survey}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=UsQpyc2AvP},
}
Figure