Introduction
Automatic Modulation Classification (AMC) is a critical component of modern wireless communication systems, enabling spectrum monitoring and interference management. While deep learning models, particularly Transformers, have significantly improved AMC performance, they remain highly susceptible to adversarial attacks. These attacks involve small, intentionally crafted perturbations that can lead a model to misclassify signals, posing a severe security risk in hostile electronic environments.
The Robustness Gap in IoT
Deploying robust AMC models on IoT devices presents a unique challenge. These devices are constrained by limited power and memory, necessitating the use of compact models. While large-scale Transformers can be made resilient through standard adversarial training (AT), compact versions often fail to retain this robustness even when trained with identical techniques. This "robustness gap" leaves low-power devices vulnerable to even simple adversarial techniques like the Fast Gradient Method or Projected Gradient Descent (PGD).
The ATARD Framework
To bridge this gap, the researchers proposed ATARD (ATtention-based Adversarial Robustness Distillation). Unlike traditional knowledge distillation, which typically focuses on matching the final output (logits) of a teacher and student, ATARD focuses on the internal Attention Maps.
As shown in the framework above, the process involves:
- Teacher-Student Architecture: A large, adversarially-trained Transformer acts as the teacher. The student is a significantly smaller model, reducing the parameter count from approximately 801k to 230k.
- Adversarial Attention Map (AAM) Transfer: During training, the student is exposed to adversarial examples. It is optimized to minimize a loss function that includes the difference between its own attention maps and those of the robust teacher (Loss2).
- Function Smoothing: By mimicking the teacher’s attention patterns under attack, the student learns a "smoother" decision function. This makes the model less sensitive to small input changes, forcing an attacker to use larger, more detectable perturbations to achieve a misclassification.
Experimental Evaluation and Results
The effectiveness of ATARD was evaluated using two primary benchmarks: RML2016.10a (a standard GNU Radio dataset) and RDL2021.12 (a complex dataset featuring Rayleigh and Rician fading).
Key results from the study include:
- Superior Robustness: Against strong PGD attacks at a Perturbation-to-Noise Ratio (PNR) of -10 dB, ATARD achieved roughly 13% higher accuracy than standard adversarial training and outperformed other state-of-the-art distillation methods like ARD, IAD, and AKD.
- Efficiency Gains: The student model maintained high robustness while being nearly 70% smaller than the teacher model, making it ideal for resource-constrained IoT hardware.
- Resilience in Fading Channels: In challenging Rayleigh fading scenarios, ATARD provided a 19% accuracy improvement over standard AT at a PNR of -20 dB.
Conclusion
ATARD demonstrates that preserving attention-based inter-patch relations is key to maintaining robustness in compact Transformer models. By distilling adversarial attention from a large teacher, the method provides a path toward secure and efficient radio signal classification for low-power IoT devices. Future work will focus on testing these architectures on physical hardware to evaluate real-time latency and exploring defenses against more sophisticated adaptive attacks.
Summary updated