Backdoor Attack with Imperceptible Input and Latent Modification

Khoa Doan, Yingjie Lao, Ping LiNeurIPS 2021 · OpenReview

Introduction

Deep Neural Networks (DNNs) are notoriously vulnerable to backdoor attacks. In a typical attack, an adversary "poisons" a small portion of the training data with a trigger (like a small square in the corner of an image). The resulting model works perfectly on normal data but performs a malicious action (like misclassifying a stop sign as a speed limit sign) whenever it sees that specific trigger.

While recent attacks have made these triggers "imperceptible" to humans by using subtle noise or warping, they still leave a "fingerprint" in the model’s internal feature space (the latent space). Existing defenses, such as Spectral Signature or Activation Clustering, look for these statistical anomalies to flag and remove poisoned samples.

Researchers from Baidu have developed the Wasserstein Backdoor (WB), a new class of backdoor attack that is virtually invisible to both human eyes and automated statistical defenses. By using a novel "Discriminant Sliced-Wasserstein Distance," the attack ensures that poisoned data looks identical to clean data not just in the pixels, but also in the deep mathematical latent representations inside the neural network.

Methods: Achieving Input and Latent Stealth

The authors propose the WB framework, which treats the attack as a constrained optimization problem with two main components:

  1. Input Stealth: Instead of a fixed patch, the method uses a generator (like a U-Net) to create image-specific, additive noise. This noise is constrained by an $l_\infty$ norm, ensuring the perturbations are too small for a human to notice.
  2. Latent Stealth (The Core Innovation): To hide the attack from statistical defenses, the framework minimizes the Wasserstein distance between the internal representations of clean images and poisoned images.
  3. Because calculating this distance in high-dimensional space is computationally expensive, they developed the Discriminant Sliced-Wasserstein Distance (DSWD).
  4. Standard "sliced" methods use random projections; DSWD instead uses the directions already learned by the classifier's final layer. This makes the calculation much more efficient and targeted at the exact features the model uses to distinguish classes.

The training follows a two-stage alternating procedure: first, alternating between updating the model and the trigger generator; then, fixing the trigger and fine-tuning the model to ensure high accuracy and attack success.

Datasets

The authors evaluated their method on four standard computer vision benchmarks:

  • MNIST: Handwritten digits.
  • CIFAR10: Small, low-resolution natural images across 10 classes.
  • GTSRB: German Traffic Sign Recognition Benchmark (a real-world safety-critical application).
  • TinyImagenet: A subset of the massive ImageNet dataset, representing more complex visual tasks.

Key Insights

  • The "Double-Blind" Attack: Most attacks focus on fooling either the human or the defense algorithm. WB is one of the first to successfully target both simultaneously, proving that "imperceptible" at the pixel level does not guarantee safety if the latent distributions are distinct.
  • Leveraging the Classifier's Knowledge: A key insight is using the classifier’s own weights to calculate the DSWD. Since the classifier is already trained to find the most important features, those same features are the best "slices" to use when trying to hide poisoned data among clean data.
  • Distribution Matching vs. Point Matching: Rather than trying to make one specific poisoned image look like one specific clean image, WB focuses on making the entire population of poisoned samples statistically indistinguishable from the clean population.

Key Results

  • High Effectiveness: WB achieves an Attack Success Rate (ASR) of nearly 100% across most datasets (e.g., 99% on CIFAR10 and GTSRB) while maintaining clean-data accuracy comparable to unpoisoned models.
  • Bypassing Latent Defenses: In tests against Activation Clustering, WB achieved a "Rand Index" (a measure of how easily clusters can be separated) near 0, whereas traditional attacks like BadNets score near 1.0 (perfectly separable).
Baseline Latent Space
WB Latent Space

As seen in the latent space visualizations above, baseline attacks leave clear separation between clean and backdoor samples, whereas the WB attack completely mixes the distributions, rendering clustering defenses useless.

  • Bypassing Spectral Signature: The correlation scores for clean and poisoned samples in WB were overlapping and indistinguishable, effectively neutralizing the Spectral Signature defense.
  • Robustness: The attack also bypassed input-level defenses like STRIP (which checks for prediction consistency under perturbation) and Neural Cleanse (which tries to reverse-engineer triggers).

Limitations and Future Work

  • Training Complexity: The two-stage alternating optimization is more complex and computationally intensive than standard backdoor training.
  • Hyperparameter Sensitivity: The balance between the "stealth" penalty ($\beta$) and the "attack" loss ($\alpha$) requires careful tuning to ensure the attack doesn't degrade the model's primary task performance.
  • Future Directions: The authors suggest that future work could explore applying these latent-matching techniques to other domains beyond image classification, such as Natural Language Processing (NLP) or Reinforcement Learning.

Summary updated

Abstract

Recent studies have shown that deep neural networks (DNN) are vulnerable to various adversarial attacks. In particular, an adversary can inject a stealthy backdoor into a model such that the compromised model will behave normally without the presence of the trigger. Techniques for generating backdoor images that are visually imperceptible from clean images have also been developed recently, which further enhance the stealthiness of the backdoor attacks from the input space. Along with the development of attacks, defense against backdoor attacks is also evolving. Many existing countermeasures found that backdoor tends to leave tangible footprints in the latent or feature space, which can be utilized to mitigate backdoor attacks. In this paper, we extend the concept of imperceptible backdoor from the input space to the latent representation, which significantly improves the effectiveness against the existing defense mechanisms, especially those relying on the distinguishability between clean inputs and backdoor inputs in latent space. In the proposed framework, the trigger function will learn to manipulate the input by injecting imperceptible input noise while matching the latent representations of the clean and manipulated inputs via a Wasserstein-based regularization of the corresponding empirical distributions. We formulate such an objective as a non-convex and constrained optimization problem and solve the problem with an efficient stochastic alternating optimization procedure. We name the proposed backdoor attack as Wasserstein Backdoor (WB), which achieves a high attack success rate while being stealthy from both the input and latent spaces, as tested in several benchmark datasets, including MNIST, CIFAR10, GTSRB, and TinyImagenet.
BibTeX
@inproceedings{doan2021backdoor,
  author = {Khoa Doan and Yingjie Lao and Ping Li},
  title = {{Backdoor Attack with Imperceptible Input and Latent Modification}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=2j_cut38wv},
}
Figure