Contextual Perturbation for Robust Natural Language Processing

This direction is merged into another direction: contextual token substitution. Its last version stays readable and citable here.

Version 1 of 163 papers74 conceptsfrozen 2026-10-04

Traditional methods for testing and attacking Natural Language Processing (NLP) models often relied on static resources like WordNet or simple character-level swaps. While these methods could flip a model's prediction, they frequently produced "adversarial examples" that were ungrammatical or nonsensical to human readers. If a sentence like "The movie was good" is changed to "The movie was bank" because "good" and "bank" share some obscure embedding relation, the test is invalid—the model failed because the input was garbage, not because it lacked robustness.

A new research direction has emerged that treats adversarial generation as a context-aware infilling task. By leveraging the internal knowledge of pre-trained Masked Language Models (MLMs) and Large Language Models (LLMs), researchers can now generate perturbations that are both effective at deceiving models and virtually indistinguishable from natural human writing.

The Mask and Infill Mechanism

The core intuition behind this direction is that a language model trained on billions of words already knows the "manifold" of natural language. If you remove a word from a sentence, the model can predict a set of candidates that are syntactically and semantically appropriate for that specific slot.

In frameworks like BAE: BERT-based Adversarial Examples for Text Classification and BERT-ATTACK by Linyang Li and Xipeng Qiu, the process follows a simple but powerful loop:

  1. Identify Importance: Determine which words in the input sentence most influence the target model's prediction.
  2. Mask: Replace a high-importance word with a [MASK] token.
  3. Infill: Use an MLM (like BERT or RoBERTa) to predict the top-$k$ most likely words for that mask.
  4. Filter and Select: From those $k$ candidates, select the one that most successfully "tricks" the target model while preserving the original label's meaning.

Because the MLM provides candidates based on the surrounding context, the word "bank" would only be suggested if the sentence was about finance or rivers, never as a synonym for "good." This ensures the adversarial example remains on the natural language manifold.

From Single Words to Semantic Spans

While early work focused on single-word substitution, the field has moved toward more complex transformations. Dianqi Li and colleagues introduced CLARE (Contextualized Perturbation for Textual Adversarial Attack), which expands the toolkit to include insertion and merging. Instead of just swapping words, CLARE can insert new contextually relevant tokens or merge two tokens into one, allowing for more flexible and stealthy attacks.

Similarly, Yu Cao proposed PLAT (Phrase-level Textual Adversarial Attack with Label Preservation), which moves the "mask-and-fill" logic from individual tokens to entire phrases. By replacing multi-word spans, these methods can swap idioms or entire descriptive clauses, maintaining a level of fluency that single-word methods cannot achieve. This "span-filling" approach is also central to the Rewrite and Rollback (R&R) framework by Lei Xu and Alfredo Cuesta-Infante, which uses a stochastic process to explore complex rewrites and then "rolls back" changes that damage the sentence's meaning.

Bridging the Discrete-Continuous Gap

One of the hardest problems in NLP is that text is discrete (words), while the gradients used to train models are continuous. In computer vision, you can slightly nudge a pixel's value; in NLP, you can't "slightly nudge" the word "cat."

Researchers like Lifan Yuan and Wei Wei addressed this in T-PGD (Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework). They perform adversarial perturbations in the continuous embedding space and then use an MLM "head" as a decoder. The MLM acts as a linguistic anchor, pulling the perturbed continuous vector back to the nearest valid, fluent word in the vocabulary. This allows NLP researchers to use the powerful optimization tools originally developed for image recognition.

Applications in Testing and Interpretability

This "contextual perturbation" isn't just for breaking models; it is a vital tool for understanding them.

  • Metamorphic Testing: Pinjia He and Zhendong Su developed SIT (Structure-Invariant Testing for Machine Translation), which uses MLMs to generate test cases for translation systems. If a system translates "The man is eating" correctly but fails on "The doctor is eating," SIT identifies a bug in the model's underlying logic.
  • Explanation Robustness: Models often provide "explanations" (highlighting which words were important). I. Girardi and Adam Ivankay have used contextual perturbations to show that these explanations are often brittle. By changing a single, non-essential word using an MLM, they can completely change which words the model claims are "important," even if the final prediction stays the same.

The Shift to Generative Oracles

The most recent evolution in this field involves moving from MLMs (which fill blanks) to LLMs (which can rewrite entire passages). Zimu Wang and colleagues demonstrated that LLMs can generate adversarial examples that are significantly more "invisible" to humans than previous methods.

Whether it is LexicalAT using reinforcement learning to find the best synonyms or AutoPoison using LLMs to inject stealthy backdoors into training data, the underlying principle remains the same: to truly test or train a language model, you must use the linguistic priors of another language model. By sampling from the natural language manifold rather than a static dictionary, researchers ensure that the "adversarial" in adversarial NLP refers to a model's reasoning, not its vocabulary.

Papers in this version

Showing 60 of 63 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_contextual_perturbation_for_robust_natur_403e3bb9,
  title = {Contextual Perturbation for Robust Natural Language Processing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contextual-perturbation-for-robust-natural-language-processing/rdr_403e3bb9f6274ed6bfb7c97fbd3bb93d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_contextual_perturbation_for_robust_natur_403e3bb9_v1,
  title = {Contextual Perturbation for Robust Natural Language Processing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contextual-perturbation-for-robust-natural-language-processing/rdr_403e3bb9f6274ed6bfb7c97fbd3bb93d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}