Abstract
Self-supervised learning (SSL) has revolutionized AI by enabling models to learn from vast, unlabeled datasets. However, these models are vulnerable to Trojan attacks, where malicious triggers cause targeted misclassifications. Current defenses often require white-box access or clean validation data—luxuries rarely available in Model-as-a-Service (MaaS) environments. This paper introduces TrojanDec, a novel, data-free, black-box defense mechanism. TrojanDec identifies malicious inputs by analyzing feature similarity shifts under random masking and restores compromised images using diffusion models. Our evaluation demonstrates that TrojanDec effectively neutralizes state-of-the-art attacks with minimal computational overhead and no reliance on clean data.
1. Introduction
The paradigm of pre-training large-scale encoders (e.g., CLIP) on internet-scale data has introduced a significant security vector: the Trojan attack. In these scenarios, an attacker poisons the training data or manipulates model parameters so that a specific trigger—such as a small patch—induces a predictable, incorrect feature representation. Because downstream classifiers inherit these poisoned features, the entire pipeline becomes compromised.
Existing defensive strategies typically rely on having a "clean" validation set to calibrate detection or require internal access to the model's weights (white-box access). In practical deployment, users often interact with models via APIs and lack the resources to curate trusted datasets. TrojanDec addresses this gap by providing a defense that is both data-free and compatible with black-box APIs.
2. Methodology
TrojanDec operates through a three-stage pipeline designed to detect and then neutralize the threat at inference time.
2.1 Metadata Extraction
When a test image is processed, TrojanDec generates a series of random masks. The system queries the black-box encoder to obtain feature vectors for both the original image and its various masked versions. The "metadata" for detection consists of the cosine similarity scores between the original feature vector and the masked feature vectors.
2.2 Trojan Detection via Gap Statistics
The core insight of TrojanDec lies in the statistical behavior of feature similarities:
- Clean Images: Masking a small portion of a clean image generally results in minor semantic shifts. The similarity scores remain high and form a single, tight cluster.
- Trojaned Images: If a mask covers the malicious trigger, the "Trojan behavior" is deactivated, causing the feature vector to shift drastically toward the image's true semantic meaning. This results in a bimodal distribution of similarity scores.
TrojanDec utilizes the Gap Statistic to automatically differentiate between these single-cluster (clean) and dual-cluster (Trojaned) distributions.
2.3 Image Restoration
Once an image is flagged as malicious, the system identifies the specific mask that caused the most significant drop in similarity—this mask likely covers the trigger. To "heal" the image, TrojanDec employs a Denoising Diffusion Null-Space Model (DDNM). This pre-trained generative model reconstructs the masked area based on the surrounding context, effectively removing the trigger while preserving the image's original semantic content.
3. Experimental Results
TrojanDec was evaluated across multiple datasets (CIFAR10, STL10, ImageNet100) and real-world encoders, including OpenAI’s CLIP.
- Detection Performance: The method achieved exceptional precision. On the STL10 dataset, TrojanDec recorded a False Negative Rate (FNR) of just 0.40% and a False Positive Rate (FPR) of 0.88%.
- Attack Neutralization: The Attack Success Rate (ASR) was reduced from 100% to approximately 1% across various benchmarks, while maintaining the accuracy of the model on clean inputs.
- Efficiency: Despite the use of diffusion models for restoration, the detection phase is highly efficient, requiring only 1.9 ms per image on an NVIDIA A100 GPU.
4. Discussion and Conclusion
TrojanDec represents a shift toward "self-referential" defense. By using the input image as its own reference point through masking, it bypasses the need for external validation data. While the current implementation is optimized for patch-based triggers, it sets a new standard for practical, deployable security in the SSL era.
Future work will explore the robustness of this approach against adaptive attackers who may attempt to use distributed or global triggers to evade masking-based detection. Additionally, the principles of TrojanDec—detecting bimodal feature shifts—may be applicable to other modalities such as natural language processing and audio analysis.
Summary updated