Detecting Universal Trigger's Adversarial Attack with Honeypot

Thai Le, Noseong Park, Dongwon LeearXiv 2020 · OpenReview

Abstract

Adversarial attacks in Natural Language Processing (NLP), particularly Universal Triggers (UniTrigger), pose significant threats due to their input-independence and transferability. This paper introduces DARCY (Defending universAl tRigger’s attaCk with honeYpot), a proactive defense mechanism that injects hidden "trapdoors" into neural networks. By baiting automated attack algorithms into predictable patterns, DARCY achieves up to 99% detection accuracy (TPR) with less than 1% false positives, effectively neutralizing powerful attacks with minimal impact on standard model performance.

The Universal Trigger Threat

Universal Triggers are a particularly dangerous class of adversarial threats. Unlike traditional attacks that perturb specific sentences, a universal trigger is a fixed, input-independent phrase (e.g., "zoning zoombie") that, when prepended to any input, causes a model to output a specific target class. These attacks are problematic because they generalize across unseen inputs, transfer between models, and allow attackers to launch mass attacks at zero additional computational cost once a trigger is found. Existing defenses often fail because these triggers consist of valid words that do not technically change the underlying meaning of the text but instead hijack the model's internal logic.

The DARCY Framework

DARCY operates on the principle of "bait and catch," borrowing the "honeypot" concept from cybersecurity. The framework proactively injects hidden trapdoors into NLP models to bait automated attack algorithms into using predictable patterns.

Overview of the DARCY framework: Trapdoor Search, Injection, and Detection.

The framework consists of three main stages:

  1. Trapdoor Searching: The system identifies a set of $K$ tokens (trapdoors) for each class based on fidelity (not interfering with clean text), robustness (using multiple trapdoors to prevent guessing), and class-awareness (distinctness in embedding space).
  2. Trapdoor Injection: The model is retrained with a modified objective function to create artificial local optima. This "baits" the loss landscape so that automated tools like UniTrigger are mathematically drawn toward using these pre-defined trapdoor tokens.
  3. Trapdoor Detection: A separate, lightweight binary classifier monitors inputs. If it detects the injected trapdoors being used in a way that matches the "baited" behavior, it flags the input as adversarial.

Experimental Results

The authors evaluated DARCY across five diverse public datasets: Clickbait (CB), Subjectivity (SJ), Movie Reviews (MR), Binary Sentiment Treebank (SST2), and AG News. The defense was tested against various model architectures, including CNN, RNN, and BERT.

Detection AUC across different models (CNN, RNN, BERT) and datasets.

Key results include:

  • High Detection Accuracy: DARCY achieved a True Positive Rate (TPR) of approximately 99% and a False Positive Rate (FPR) of less than 1% in "Novice" attack scenarios.
  • Resilience to Adaptive Attacks: Even when attackers are aware of the honeypot, increasing the number of trapdoors (e.g., $K \ge 5$) significantly reduces the probability of bypassing the defense.
  • Minimal Performance Tax: DARCY maintains high performance on benign data, with only a ~2% average drop in F1 score.

Discussion and Conclusion

DARCY successfully turns an attacker's strength—the reliance on optimization to find the "easiest" adversarial path—into a weakness by intentionally making that path lead to a trap. The use of multiple trapdoors provides a robust defense against sophisticated attackers. While highly effective against UniTrigger attacks, future work may explore generalizing this honeypot approach to other generative adversarial attacks and further investigating its behavior in massive transformer-based models.

Summary updated

BibTeX
@misc{le2020detecting,
  author = {Thai Le and Noseong Park and Dongwon Lee},
  title = {{Detecting Universal Trigger's Adversarial Attack with Honeypot}},
  year = {2020},
  url = {https://openreview.net/forum?id=NwkQrWCC61C},
}
Figure