PiDAn: A Coherence Optimization Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks

Yue Wang, Wenqing Li, Esha Sarkar, Muhammad Shafique, Michail Maniatakos, Saif Eddin Jabari

Abstract

Deep Neural Networks (DNNs) are increasingly vulnerable to backdoor attacks, where malicious triggers embedded in training data cause misclassification at inference time. This paper presents PiDAn, a defense framework that leverages coherence optimization to identify poisoned samples. By analyzing internal neural representations, PiDAn distinguishes between authentic and poisoned data based on their membership in distinct linear subspaces. Experimental results on GTSRB and ImageNet demonstrate that PiDAn achieves over 90% detection accuracy and 95% identification precision, outperforming state-of-the-art methods against stealthy hidden-trigger attacks.

Introduction

Backdoor attacks pose a significant threat to the security of deep learning systems. In these scenarios, an adversary poisons a small fraction of the training set with a specific trigger. While the model maintains high accuracy on clean inputs, it reliably misclassifies any input containing the trigger into a target class. Modern "Hidden Trigger" attacks are particularly insidious, as poisoned images appear natural to human observers and statistically similar to the target class, bypassing traditional anomaly detectors. PiDAn addresses this challenge by focusing on the fundamental properties of neural representations rather than specific attack signatures.

Methodology: Coherence Optimization and the Grouping Effect

PiDAn is based on the observation that while poisoned and clean data may share a label, their internal neural activations occupy different linear subspaces within the network's final layers.

Coherence Optimization

The framework formulates a convex optimization problem to learn sample-wise weights for a given class. The goal is to maximize "projected coherence," a metric that highlights structural differences between the subspaces of clean and poisoned data.

The Grouping Effect and Detection

Mathematically, these optimized weights exhibit a "grouping effect," where authentic and poisoned samples cluster into distinct weight distributions. This creates a measurable gap that can be analyzed using a Likelihood Ratio Test. By fitting the weights to both a single Gaussian (representing a clean class) and a two-component Gaussian mixture (representing a poisoned class), PiDAn can flag infected classes with high confidence.

Separation of authentic and poisoned data through subspace analysis (a) and weight optimization (b).

Mitigation

Once a class is flagged, K-means clustering is applied to the learned weights to isolate poisoned samples. These samples are then removed or relabeled, allowing the model to be retrained on a purified dataset.

Experimental Evaluation

The effectiveness of PiDAn was evaluated against three major attack types: Hidden Trigger, TaCT, and Badnets.

Datasets

Testing was conducted on two primary benchmarks:

  • GTSRB (German Traffic Sign Recognition Benchmark): A dataset critical for autonomous driving safety.
  • ILSVRC2012 (ImageNet): A large-scale dataset used to verify scalability and performance on complex object recognition.
Example of a traffic sign from the GTSRB dataset used for evaluating backdoor detection.

Key Results

PiDAn demonstrated superior performance compared to existing methods like Activation Clustering and Spectral Signatures. Key metrics include:

  • Detection Accuracy: Successfully identified over 90% of infected classes.
  • Identification Precision: Isolated 95% of poisoned samples, enabling near-perfect dataset purification.
  • Efficiency: The algorithm is highly efficient, processing the entire GTSRB dataset in approximately 6 seconds.

Conclusion

PiDAn provides a robust, attack-agnostic defense against sophisticated backdoor attacks. By leveraging the "flattening" property of deep networks—where complex manifolds are reduced to linear subspaces—PiDAn effectively unmasks stealthy triggers that evade traditional feature-space analysis. While the method requires a small set of clean samples for feature centralization, its high precision and computational efficiency make it a practical solution for securing large-scale DNN training pipelines.

Summary updated

Abstract

Backdoor attacks impose a new threat in Deep Neural Networks (DNNs), where a backdoor is inserted into the neural network by poisoning the training dataset, misclassifying inputs that contain the adversary trigger. The major challenge for defending against these attacks is that only the attacker knows the secret trigger and the target class. The problem is further exacerbated by the recent introduction of "Hidden Triggers", where the triggers are carefully fused into the input, bypassing detection by human inspection and causing backdoor identification through anomaly detection to fail. To defend against such imperceptible attacks, in this work we systematically analyze how representations, i.e., the set of neuron activations for a given DNN when using the training data as inputs, are affected by backdoor attacks. We propose PiDAn, an algorithm based on coherence optimization purifying the poisoned data. Our analysis shows that representations of poisoned data and authentic data in the target class are still embedded in different linear subspaces, which implies that they show different coherence with some latent spaces. Based on this observation, the proposed PiDAn algorithm learns a sample-wise weight vector to maximize the projected coherence of weighted samples, where we demonstrate that the learned weight vector has a natural "grouping effect" and is distinguishable between authentic data and poisoned data. This enables the systematic detection and mitigation of backdoor attacks. Based on our theoretical analysis and experimental results, we demonstrate the effectiveness of PiDAn in defending against backdoor attacks that use different settings of poisoned samples on GTSRB and ILSVRC2012 datasets. Our PiDAn algorithm can detect more than 90% infected classes and identify 95% poisoned samples.
BibTeX
@article{wang2024pidan,
  author = {Yue Wang and Wenqing Li and Esha Sarkar and Muhammad Shafique and Michail Maniatakos and Saif Eddin Jabari},
  title = {{PiDAn: A Coherence Optimization Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks}},
  journal = {IEEE Trans. Artif. Intell.},
  year = {2024},
  doi = {10.1109/tai.2024.3373720},
}
Figure