Measure and Improve Robustness in NLP Models: A Survey

Xuezhi Wang, Haohan Wang, Diyi Yang

Abstract

As NLP models achieved state-of-the-art performances over benchmarks and gained wide applications, it has been increasingly important to ensure the safe deployment of these models in the real world, e.g., making sure the models are robust against unseen or challenging scenarios. Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research. In this paper, we aim to provide a unifying survey of how to define, measure and improve robustness in NLP. We first connect multiple definitions of robustness, then unify various lines of work on identifying robustness failures and evaluating models' robustness. Correspondingly, we present mitigation strategies that are data-driven, model-driven, and inductive-prior-based, with a more systematic view of how to effectively improve robustness in NLP models. Finally, we conclude by outlining open challenges and future directions to motivate further research in this area.

Introduction

While Natural Language Processing (NLP) models have reached state-of-the-art performance on standard benchmarks, they remain notoriously "brittle." A model that achieves 90% accuracy on a curated dataset might plummet when faced with a simple typo, a slight change in dialect, or a distracting sentence that does not actually change the semantic meaning.

This survey addresses the fragmented nature of robustness research. Historically, "adversarial robustness" (defending against intentional attacks) and "out-of-distribution generalization" (handling natural changes in data) were treated as separate problems. This paper argues they are two sides of the same coin: a failure of the model to generalize from its training distribution ($D$) to a shifted test distribution ($D'$). The core challenge is that models often learn "shortcuts" or spurious correlations—patterns that happen to work on the training set but are not logically connected to the actual task.

Taxonomy of Robustness Methods

The survey establishes a comprehensive taxonomy for the field, categorizing research into three main pillars: definitions, identification of failures, and mitigation strategies. Robustness is defined as the performance gap between training data and shifted data, whether that shift is synthetic (e.g., character swaps, synonym replacement) or natural (e.g., different domains or grammatical errors).

Methods for generating these shifts vary based on the space (discrete vs. continuous) and the level of perturbation, as shown in the table below:

Taxonomy of adversarial attack methods across discrete and continuous spaces.

Failures are identified through two primary lenses:

  • Human-driven: Using error analysis and linguistic priors to create "stress tests" (e.g., checking if an NLI model relies solely on the hypothesis).
  • Model-driven: Using "bias-only" models to find what a low-capacity model can solve, which often reveals the "shortcuts" in the dataset.

Robustness Benchmarks

To evaluate these failures, the paper provides a curated list of robustness benchmarks designed to break standard models across various NLP tasks. These benchmarks move beyond standard accuracy to test specific linguistic heuristics and adversarial vulnerabilities.

Summary of robustness benchmarks across various NLP tasks.

Key examples include HANS for Natural Language Inference (testing heuristics), AdvSQuAD for Question Answering (adding distracting sentences), and PAWS for Paraphrase Identification (word scrambling that preserves syntax but changes meaning).

Mitigation Strategies

Improving robustness requires a multi-faceted approach:

  • Data-driven: Techniques like Mixup, counterfactual data augmentation, and strategic "shortcut" perturbation.
  • Model/Training-based: Leveraging massive pre-training and focusing on "minority examples" (hard-to-learn cases).
  • Inductive-prior-based: Adding regularizers that force the model to ignore known biases, such as ensembling with a bias-only model.
  • Causal Intervention: Using causal structures to ensure models are invariant to non-causal features.

Key Insights and Results

  • The "Shortcut" Problem: Models exhibit a lack of robustness because they are "lazy," utilizing spurious correlations (e.g., associating the word "not" with a "contradiction" label) rather than learning underlying logic.
  • NLP vs. Vision: Unlike computer vision, where perturbations are often imperceptible pixel noise, NLP perturbations are usually perceptible but meaning-preserving. The discrete nature of text makes gradient-based attacks significantly harder.
  • The Robustness-Accuracy Trade-off: Removing shortcuts can sometimes lead to a slight drop in standard benchmark performance while significantly increasing real-world reliability.
  • Pre-training Benefits: Large-scale pre-training (e.g., BERT, GPT) inherently improves out-of-distribution robustness compared to training from scratch, likely due to the diverse "support" of language seen during pre-training.

Future Directions

The authors identify several open questions for the community. First, there is a need for automatic discovery of model weaknesses to reduce reliance on human intuition. Second, the field requires end-to-end frameworks that are robust by design rather than relying on pipeline-based bias detection. Finally, a unified evaluation standard, such as CheckList or Dynabench, is essential for consistent progress across the research community.

Summary updated

Abstract

As NLP models achieved state-of-the-art performances over benchmarks and gained wide applications, it has been increasingly important to ensure the safe deployment of these models in the real world, e.g., making sure the models are robust against unseen or challenging scenarios. Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research. In this paper, we aim to provide a unifying survey of how to define, measure and improve robustness in NLP. We first connect multiple definitions of robustness, then unify various lines of work on identifying robustness failures and evaluating models' robustness. Correspondingly, we present mitigation strategies that are data-driven, model-driven, and inductive-prior-based, with a more systematic view of how to effectively improve robustness in NLP models. Finally, we conclude by outlining open challenges and future directions to motivate further research in this area.
BibTeX
@inproceedings{wang2022measure,
  author = {Xuezhi Wang and Haohan Wang and Diyi Yang},
  title = {{Measure and Improve Robustness in NLP Models: A Survey}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=UsQpyc2AvP},
}
Figure