Vision Transformer with Adversarial Indicator Token against Adversarial Attacks in Radio Signal Classifications
Abstract
Deep learning models for Automatic Modulation Classification (AMC) are increasingly vulnerable to adversarial attacks, where subtle perturbations can lead to catastrophic misclassification. This paper presents AiTViT, a novel Vision Transformer architecture designed to simultaneously classify radio signals and detect adversarial tampering. By introducing a specialized Adversarial Indicator (AdvI) token, the model achieves state-of-the-art robustness and efficiency, making it suitable for resource-constrained IoT and battlefield communication systems.
Introduction
Automatic Modulation Classification (AMC) serves as a foundational component in modern wireless communication, enabling autonomous spectrum management and signal interception. This is critical for managing spectrum in massive IoT networks or intercepting communications in battlefield scenarios. However, the reliance on deep neural networks introduces a significant security risk: adversarial attacks. These attacks involve injecting carefully crafted, low-power noise into signals to deceive classification models, often making them invisible to traditional filters.
Traditional defense mechanisms often rely on multi-stage pipelines—typically a separate detector followed by a classifier. Such approaches are computationally prohibitive for small, battery-powered IoT devices. AiTViT addresses this by integrating detection and classification into a single, unified Transformer backbone.
The AiTViT Architecture
The core innovation of AiTViT is the Adversarial Indicator (AdvI) token. While a standard Vision Transformer (ViT) uses a classification (CLS) token to aggregate global information for label prediction, AiTViT appends a second learnable embedding: the AdvI token.
Dual-Purpose Processing and Attention
The model processes input signals through a convolutional layer for initial feature extraction, followed by a series of Transformer encoder layers. Within these layers, the self-attention mechanism allows the AdvI token to interact with all signal patches.
Visualization of the attention mechanism reveals that while standard Transformers spread their attention across the whole signal, the AiTViT model’s attention is highly concentrated. The AdvI token acts like a spotlight, pinpointing the exact features (phase, amplitude, or frequency shifts) where an adversary injected malicious noise.
Unified Training and Defense
AiTViT employs a joint loss function that balances classification accuracy and attack detection accuracy. This unified approach allows the model to perform both tasks in a single forward pass, drastically reducing latency compared to ensemble-based defenses like "Neural Rejection" methods that rely on separate Support Vector Machines (SVMs).
Experimental Evaluation
The researchers evaluated AiTViT using two benchmark datasets:
- RML2016.10a: A standard GNU Radio dataset containing 220,000 entries across 11 modulation types.
- RDL2021.12: A more challenging dataset that includes complex real-world conditions like Rician fading and Alpha-stable (impulsive) noise.
To simulate realistic threats, the authors developed an adaptive version of the Projected Gradient Descent (PGD) attack. This method scales the perturbation size based on the Signal-to-Noise Ratio (SNR), ensuring the attacks remain "imperceptible" relative to the background noise.
Key Results and Discussion
AiTViT demonstrated significant performance gains over baselines like standard Adversarial Training (AT) and Neural Rejection (NR):
- Robustness: In high-interference environments (PNR = -10 dB), AiTViT achieved approximately 10% higher accuracy than AT-trained Transformers and 26% higher than NR systems.
- Detection Accuracy: Under strong attacks (PNR = 0 dB), the model’s False Negative Rate was only 32%, whereas baseline methods failed nearly 100% of the time.
- Efficiency: The model maintained a low CPU utilization (1.4%) and memory footprint (1568 MB), achieving a processing time of 0.21 seconds for 1,000 samples.
Conclusion and Future Work
AiTViT represents a significant step forward in securing wireless communications. By leveraging the attention mechanism of Transformers through a dedicated indicator token, it provides a lightweight yet powerful defense against adversarial perturbations. Future work will explore the model's performance against "black-box" transfer attacks and its adaptability to even more diverse, non-stationary channel environments.
Summary updated