ONION: A Simple and Effective Defense Against Textual Backdoor Attacks

Fanchao Qi, Yangyi Chen, Mukai Li, Zhiyuan Liu, Maosong Sun

Introduction

Deep Neural Networks (DNNs) are increasingly vulnerable to backdoor attacks, where an adversary "poisons" a model to behave normally on clean data but output a target label when a specific "trigger" is present. While defenses in computer vision are maturing, NLP has lagged behind. Most existing textual defenses are limited to the pre-training phase, requiring access to training data. However, in modern AI workflows, users often download pre-trained models or use third-party APIs. ONION (backdOor defeNse with outlIer wOrd detectioN) addresses this gap by providing a model-agnostic, post-training defense that identifies and removes triggers during inference.

The ONION Method: Leveraging the Fluency Gap

The core intuition behind ONION is that textual triggers are typically "context-free." When inserted into a sentence, they act as outliers that disrupt the natural linguistic flow. ONION quantifies this disruption using perplexity, a measure of how "surprising" a sentence is to a language model (e.g., GPT-2).

Suspicion Score Calculation

For an input sentence, ONION calculates a Suspicion Score ($f_i$) for each word $w_i$ by measuring the change in perplexity when that word is removed: $$f_i = p_0 - p_i$$ where $p_0$ is the perplexity of the original sentence and $p_i$ is the perplexity after removing $w_i$. If removing a word significantly decreases perplexity (making the sentence more fluent), it is flagged as a potential backdoor trigger.

Figure 1: Distribution of suspicion scores for trigger words versus normal words. Trigger words consistently exhibit higher scores, indicating they disrupt sentence fluency.

If $f_i$ exceeds a threshold $t_s$, the word is removed before the sentence is processed by the victim model. This simple approach proved as effective as more complex search algorithms while remaining significantly faster.

Experimental Evaluation

The authors evaluated ONION across three benchmarks: SST-2 (sentiment analysis), OffensEval (offensive language detection), and AG News (topic classification). They tested the defense against various attack vectors, including BadNet (rare/common word insertion), RIPPLES (weight poisoning), and InSent (sentence-level triggers).

Key Results

ONION demonstrates remarkable effectiveness in reducing the Attack Success Rate (ASR) while maintaining high Clean Accuracy (CACC).

Figure 5: Performance of ONION across different datasets and models. The table highlights significant reductions in ASR (ΔASR) with minimal impact on clean accuracy (ΔCACC).
  • Effectiveness: ONION reduced the ASR by an average of over 40% across all tested models (BiLSTM and BERT).
  • Minimal Utility Loss: The impact on CACC was negligible, with an average drop of only 0.99%.
  • Robustness: The defense is effective even when the exact trigger is unknown, as removing just a portion of a multi-word trigger is often sufficient to disarm the backdoor.

Conclusion and Future Work

ONION is the first effective defense for the common "plug-and-play" scenario where users receive pre-backdoored models. By focusing on the linguistic "fluency gap" created by triggers, it provides a simple yet powerful barrier against insertion-based attacks. Future research will focus on extending these principles to counter more sophisticated, context-aware, or syntactic triggers that aim to blend seamlessly into the natural language distribution.

Summary updated

BibTeX
@inproceedings{qi2021onion,
  author = {Fanchao Qi and Yangyi Chen and Mukai Li and Zhiyuan Liu and Maosong Sun},
  title = {{ONION: A Simple and Effective Defense Against Textual Backdoor Attacks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.752},
}
Figure