Defending Against Backdoor Attacks by Layer-wise Feature Analysis (Extended Abstract)

Najeeb Jebreel, Josep Domingo‐Ferrer, Yiming Li, Qi Zhang, Ruichen Cao, Liang He2024 · DOI

Abstract

Deep neural networks (DNNs) are increasingly vulnerable to backdoor attacks, where adversaries poison training data to embed hidden triggers. While traditional defenses analyze feature representations in the final layer, this approach often fails because poisoned samples become indistinguishable from benign ones by the end of the network. This paper introduces a defense based on Layer-wise Feature Analysis. By identifying a "Layer of Interest" (LOI) where the network's internal representations of benign and poisoned samples diverge most significantly, we develop a robust filtering mechanism. Our method, dubbed DBALFA, requires no retraining and achieves near-perfect detection rates on standard benchmarks like CIFAR-10 and GTSRB, significantly outperforming current state-of-the-art defenses.

Introduction

The high cost of training deep learning models has led to a widespread reliance on third-party datasets and pre-trained models. This supply chain dependency introduces a critical security risk: backdoor attacks. In these attacks, a model is trained on a small percentage of poisoned data containing a "trigger" (e.g., a specific pixel pattern). The model performs normally on clean data but misclassifies any input containing the trigger into a target class chosen by the attacker.

Current detection methods often focus on the feature space of the final layer. However, this paper argues that the final layer is a bottleneck. As a sample progresses through the network, the model's objective is to map it to a specific class. By the final layer, the network has often successfully "forced" a poisoned sample to look like its target class, making detection nearly impossible.

Methodology: Locating the Layer of Interest

The core of the proposed defense is identifying the optimal layer for inspection, termed the Layer of Interest (LOI).

Identifying the LOI

Instead of analyzing every layer or just the last one, the method uses a small set of clean validation data to track the "class-like" progression of features. The LOI is defined as the layer where the model makes the most significant progress in correctly identifying benign samples, typically indicated by a sharp increase in cosine similarity to the class centroid.

Cosine similarity across layers, highlighting the Layer of Interest (LOI) where the gap between benign and poisoned samples is most pronounced.

Inference-time Filtering

During inference, the system acts as a firewall. For any incoming sample, the model calculates its cosine similarity to the benign centroids at the LOI and its immediate predecessors. If the summed similarity is significantly lower than the expected baseline for benign samples, the input is flagged as poisoned and blocked. This "plug-and-play" approach is efficient and requires no modification to the underlying model architecture.

Experimental Evaluation

The researchers evaluated the defense on two major computer vision benchmarks: CIFAR-10 and the German Traffic Sign Recognition Benchmark (GTSRB). They tested against various attack vectors, including BadNets (visible patches) and Blended (invisible noise).

Feature distributions for BadNets and Blended attacks on CIFAR10-ResNet18, showing the distinct clusters formed by benign (green) and poisoned (red) samples.

Key Results

  • Detection Accuracy: On CIFAR-10, the method achieved an average True Positive Rate (TPR) of 98.09% with a False Positive Rate (FPR) of only 1.34%.
  • Robustness on GTSRB: While many existing defenses fail on the complex GTSRB dataset, this layer-wise approach maintained a 100% TPR across nearly all tested attack types.
  • Efficiency: The reliance on simple cosine similarity metrics makes the defense suitable for real-time applications.

Discussion: The "Last Layer" Myth

A critical insight from this work is that the layer right before the classifier is often the least effective for defense. The "critical transition" where poisoned samples reveal themselves usually occurs in the second half of the network—after features become class-specific but before the backdoor trigger fully activates the final classification. By using benign behavior as a map, one can pinpoint this location without needing prior knowledge of the attack type.

Conclusion

Layer-wise Feature Analysis provides a powerful and efficient way to defend against backdoor attacks. By shifting the focus from the final layer to the "Layer of Interest," this method exposes the internal "fingerprint" left by poisoned samples. Future work will explore the applicability of this technique to other domains, such as Natural Language Processing and speech recognition.

Summary updated

BibTeX
@misc{jebreel2024defending,
  author = {Najeeb Jebreel and Josep Domingo‐Ferrer and Yiming Li and Qi Zhang and Ruichen Cao and Liang He},
  title = {{Defending Against Backdoor Attacks by Layer-wise Feature Analysis (Extended Abstract)}},
  year = {2024},
  doi = {10.24963/ijcai.2024/933},
}
Figure