A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger's Adversarial Attacks

Thai Le, Noseong Park, Dongwon Lee

Abstract

The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial textual attack method. Utilizing a learning-based mechanism, UniTrigger generates a fixed phrase that, when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. To defend against this attack that can cause significant harm, in this paper, we borrow the "honeypot" concept from the cybersecurity community and propose DARCY, a honeypot-based defense framework against UniTrigger. DARCY greedily searches and injects multiple trapdoors into an NN model to "bait and catch" potential attacks. Through comprehensive experiments across four public datasets, we show that DARCY detects UniTrigger's adversarial attacks with up to 99% TPR and less than 2% FPR in most cases, while maintaining the prediction accuracy (in F1) for clean inputs within a 1% margin. We also demonstrate that DARCY with multiple trapdoors is also robust to a diverse set of attack scenarios with attackers' varying levels of knowledge and skills.

Introduction

Adversarial attacks in Natural Language Processing (NLP) traditionally involve perturbing specific inputs to fool a model. However, a more dangerous class of "learning-based" attacks has emerged, most notably the Universal Trigger (UniTrigger). A UniTrigger is a short, fixed phrase (e.g., "zoning zoombie") that an attacker can append to any input to force a specific model output, such as making every movie review appear "Negative." These triggers are found by optimizing over an entire dataset to exploit a model's global weaknesses. Because they are computationally cheap to deploy and highly transferable, they pose a significant threat to production systems. Existing defenses often fail because these triggers are specifically optimized to "blend in" to the model's internal decision logic.

Methodology

The proposed framework, DARCY (Defending universAl tRigger’s attaCk with honeYpot), borrows the "honeypot" concept from traditional cybersecurity. Instead of merely attempting to find attacks, DARCY proactively "baits" the attacker's optimization process.

Overview of the DARCY framework for trapdoor search, injection, and detection.

The framework operates in three main stages:

  1. Searching Trapdoors: DARCY uses a "Greedy Trapdoor Search" to identify tokens that are naturally distant from a target class's decision boundary but close to each other. Selection criteria include fidelity (maintaining model accuracy), robustness (resistance to being ignored), and class-awareness.
  2. Injecting Trapdoors: The target model is retrained using a combination of clean data and "trapdoor-embedded" examples. This intentionally creates artificial local optima (the "honeypots") in the model's loss landscape. When an attacker runs the UniTrigger optimization, they are naturally drawn into these pre-defined trapdoors.
  3. Detecting Attacks: A small binary classifier is trained to monitor the model's internal representations (latent features). If an input triggers one of the injected honeypots, the detector flags it as adversarial.

Experimental Setup

DARCY was evaluated across four diverse public datasets:

  • Subjectivity (SJ): Classifying text as subjective or objective.
  • Movie Reviews (MR): Binary sentiment analysis.
  • Binary Sentiment Treebank (SST): Fine-grained sentiment analysis.
  • AG News: A 4-class news topic classification dataset.

These datasets cover various scales and tasks, providing a robust testing ground across popular NLP architectures, including CNNs, RNNs, and BERT.

Key Results

DARCY demonstrates near-perfect detection capabilities. Under "Novice" attack scenarios, it achieved a True Positive Rate (TPR) of up to 99% with a False Positive Rate (FPR) of less than 2%. Even against "Advanced Adaptive" attackers, DARCY maintained a detection AUC of over 90% in most cases when utilizing multiple trapdoors ($K \ge 5$).

Detection AUC across different model architectures and datasets.

Crucially, DARCY maintains clean-input F1 scores within a 1% margin of the original model, proving that honeypots can be effectively "silent" until triggered by an adversarial input.

Discussion and Conclusion

DARCY successfully turns the attacker's strength—optimization—into a weakness by poisoning the loss landscape with detectable "rabbit holes." The use of multiple trapdoors significantly enhances robustness against adaptive attackers.

However, limitations remain. An "Oracle Attack," where the attacker has full access to both the target model and the detection network, can still bypass the defense. Future work will focus on extending this honeypot logic to other adversarial generators, such as those utilizing paraphrasing or synonym substitution, and improving transferability across all datasets.

Summary updated

Abstract

The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial textual attack method. Utilizing a learning-based mechanism, UniTrigger generates a fixed phrase that, when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. To defend against this attack that can cause significant harm, in this paper, we borrow the "honeypot" concept from the cybersecurity community and propose DARCY, a honeypot-based defense framework against UniTrigger. DARCY greedily searches and injects multiple trapdoors into an NN model to "bait and catch" potential attacks. Through comprehensive experiments across four public datasets, we show that DARCY detects UniTrigger's adversarial attacks with up to 99% TPR and less than 2% FPR in most cases, while maintaining the prediction accuracy (in F1) for clean inputs within a 1% margin. We also demonstrate that DARCY with multiple trapdoors is also robust to a diverse set of attack scenarios with attackers' varying levels of knowledge and skills. Source code will be released upon the acceptance of this paper.
BibTeX
@inproceedings{le2021sweet,
  author = {Thai Le and Noseong Park and Dongwon Lee},
  title = {{A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger's Adversarial Attacks}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.296},
}
Figure