Introduction
Deep Neural Networks (DNNs) are increasingly vulnerable to backdoor attacks, where an adversary "poisons" a model to behave normally on clean data but output a target label when a specific "trigger" is present. While defenses in computer vision are maturing, NLP has lagged behind. Most existing textual defenses are limited to the pre-training phase, requiring access to training data. However, in modern AI workflows, users often download pre-trained models or use third-party APIs. ONION (backdOor defeNse with outlIer wOrd detectioN) addresses this gap by providing a model-agnostic, post-training defense that identifies and removes triggers during inference.
The ONION Method: Leveraging the Fluency Gap
The core intuition behind ONION is that textual triggers are typically "context-free." When inserted into a sentence, they act as outliers that disrupt the natural linguistic flow. ONION quantifies this disruption using perplexity, a measure of how "surprising" a sentence is to a language model (e.g., GPT-2).
Suspicion Score Calculation
For an input sentence, ONION calculates a Suspicion Score ($f_i$) for each word $w_i$ by measuring the change in perplexity when that word is removed: $$f_i = p_0 - p_i$$ where $p_0$ is the perplexity of the original sentence and $p_i$ is the perplexity after removing $w_i$. If removing a word significantly decreases perplexity (making the sentence more fluent), it is flagged as a potential backdoor trigger.
If $f_i$ exceeds a threshold $t_s$, the word is removed before the sentence is processed by the victim model. This simple approach proved as effective as more complex search algorithms while remaining significantly faster.
Experimental Evaluation
The authors evaluated ONION across three benchmarks: SST-2 (sentiment analysis), OffensEval (offensive language detection), and AG News (topic classification). They tested the defense against various attack vectors, including BadNet (rare/common word insertion), RIPPLES (weight poisoning), and InSent (sentence-level triggers).
Key Results
ONION demonstrates remarkable effectiveness in reducing the Attack Success Rate (ASR) while maintaining high Clean Accuracy (CACC).
- Effectiveness: ONION reduced the ASR by an average of over 40% across all tested models (BiLSTM and BERT).
- Minimal Utility Loss: The impact on CACC was negligible, with an average drop of only 0.99%.
- Robustness: The defense is effective even when the exact trigger is unknown, as removing just a portion of a multi-word trigger is often sufficient to disarm the backdoor.
Conclusion and Future Work
ONION is the first effective defense for the common "plug-and-play" scenario where users receive pre-backdoored models. By focusing on the linguistic "fluency gap" created by triggers, it provides a simple yet powerful barrier against insertion-based attacks. Future research will focus on extending these principles to counter more sophisticated, context-aware, or syntactic triggers that aim to blend seamlessly into the natural language distribution.
Summary updated