TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networks

Khondoker Murad Hossain, Tim Oates

Abstract

As deep neural networks and the datasets used to train them get larger, the default approach to integrating them into research and commercial projects is to download a pre-trained model and fine tune it. But these models can have uncertain provenance, opening up the possibility that they embed hidden malicious behavior such as trojans or backdoors, where small changes to an input (triggers) can cause the model to produce incorrect outputs (e.g., to misclassify). This paper introduces a novel approach to backdoor detection that uses two tensor decomposition methods applied to network activations. This has a number of advantages relative to existing detection methods, including the ability to analyze multiple models at the same time, working across a wide variety of network architectures, making no assumptions about the nature of triggers used to alter network behavior, and being computationally efficient. We provide a detailed description of the detection pipeline along with results on models trained on the MNIST digit dataset, CIFAR-10 dataset, and two difficult datasets from NIST's TrojAI competition. These results show that our method detects backdoored networks more accurately and efficiently than current state-of-the-art methods.

Introduction

In the modern AI landscape, the standard operating procedure for many researchers and developers is to download pre-trained models from public hubs and fine-tune them for specific tasks. However, this reliance on third-party models introduces a significant security risk: backdoors. A backdoored model functions normally on standard inputs but produces malicious outputs when triggered by a specific, often subtle, pattern known as a trojan trigger.

Examples of poisoned vs. clean samples across MNIST, TrojAI, and CIFAR-10 datasets.

The Backdoor Threat

Detecting these "trojaned" models is exceptionally challenging because the trigger is unknown to the defender. Traditional methods often attempt to reverse-engineer the trigger through computationally expensive optimization processes or rely on narrow assumptions about the trigger's appearance. These approaches struggle to scale as models grow in complexity and datasets expand, often requiring significant time and compute resources to vet a single model.

Methodology: The TEN-GUARD Pipeline

TEN-GUARD shifts the detection focus from model weights to internal activations. The core hypothesis is that backdoored models exhibit unique, shared statistical patterns in their internal representations when processing data—patterns that clean models do not share. By identifying these shared latent structures, TEN-GUARD can flag suspicious models without needing to know the specific trigger used.

The TEN-GUARD pipeline: Feature extraction via Random Projection followed by detection using IVA or PARAFAC2 tensor decomposition.

The pipeline consists of three primary stages:

  1. Feature Extraction: Activations are captured from the final hidden layer of a set of pre-trained models. Random Projection (RP) is employed to compress these high-dimensional signals into a uniform, memory-efficient size, ensuring the method remains model-agnostic.
  2. Tensor Decomposition: Advanced techniques—Independent Vector Analysis (IVA) and Parallel Factor Analysis (PARAFAC2)—are applied to the activation data to identify common "sources" or patterns across multiple models.
  3. Detection: By analyzing the resulting mixing matrices or cross-correlations, the system identifies clusters. Trojaned models tend to occupy the same subspace because they react to triggers in statistically similar ways. If a test model shows significant correlation with known backdoored patterns, it is flagged.

Experimental Results

The researchers evaluated TEN-GUARD against several benchmarks, including MNIST, CIFAR-10, and the rigorous NIST TrojAI datasets which feature complex architectures like ResNet50 and DenseNet121.

Accuracy and Efficiency

TEN-GUARD (specifically the PARAFAC2 variant) demonstrated superior performance compared to state-of-the-art methods like Neural Cleanse (NC) and K-Arm. On the complex TrojAI ResNet50 dataset, TEN-GUARD achieved an ROC-AUC of 0.89, surpassing existing optimization-based tools.

Crucially, TEN-GUARD offers a massive leap in efficiency. While optimization-based methods like ULP required nearly 4,000 seconds to process the TrojAI datasets, TEN-GUARD completed the task in approximately 240 seconds—a 15x speedup. This efficiency makes it suitable for integration into real-time MLOps pipelines where rapid vetting of models is essential.

Conclusion

TEN-GUARD provides a scalable, model-agnostic solution for vetting third-party neural networks. By leveraging the shared subspace signals of trojaned models through tensor decomposition, it achieves high detection accuracy with significantly lower computational overhead than existing optimization-based defenses. Future work will explore multi-layer activation analysis and the robustness of the system against adaptive attacks designed to mask these subspace patterns.

Summary updated

Abstract

As deep neural networks and the datasets used to train them get larger, the default approach to integrating them into research and commercial projects is to download a pre-trained model and fine tune it. But these models can have uncertain provenance, opening up the possibility that they embed hidden malicious behavior such as trojans or backdoors, where small changes to an input (triggers) can cause the model to produce incorrect outputs (e.g., to misclassify). This paper introduces a novel approach to backdoor detection that uses two tensor decomposition methods applied to network activations. This has a number of advantages relative to existing detection methods, including the ability to analyze multiple models at the same time, working across a wide variety of network architectures, making no assumptions about the nature of triggers used to alter network behavior, and being computationally efficient. We provide a detailed description of the detection pipeline along with results on models trained on the MNIST digit dataset, CIFAR-10 dataset, and two difficult datasets from NIST's TrojAI competition. These results show that our method detects backdoored networks more accurately and efficiently than current state-of-the-art methods.
BibTeX
@inproceedings{hossain2024ten,
  author = {Khondoker Murad Hossain and Tim Oates},
  title = {{TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networks}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=qvyEjTPCID},
}
Figure