A Deep Ensemble-Based Wireless Receiver Architecture for Mitigating Adversarial Attacks in Automatic Modulation Classification

Rajeev Sahay, Christopher G. Brinton, David J. LoveIEEE Trans. Cogn. Commun. Netw 2022 · OpenReview

TL;DR

This paper introduces a novel wireless receiver architecture designed to defend against adversarial attacks in Automatic Modulation Classification (AMC). By leveraging an ensemble of deep learning models trained on both time and frequency domain signal representations, their proposed "Assorted Deep Ensemble" (ADE) defense significantly improves classification accuracy in the face of sophisticated, transferable adversarial interference.

What Problem Does This Paper Address?

Modern wireless communication systems rely heavily on Automatic Modulation Classification (AMC) to efficiently manage the increasingly crowded radio spectrum. Deep learning (DL) models have shown great promise in achieving high AMC accuracy without manual feature engineering. However, these DL models are vulnerable to "adversarial attacks," where an adversary injects carefully crafted interference signals that cause the DL model to misclassify the modulation scheme. A critical challenge is that these adversarial attacks are often "transferable," meaning an attack designed for one DL model can also fool other, different DL models. This "black box" attack scenario, where the adversary has limited knowledge of the target system, poses a significant threat to the security and reliability of wireless communications. The paper aims to address this by developing a robust defense mechanism against such transferable adversarial attacks.

Methods

The core of the proposed solution is the Assorted Deep Ensemble (ADE) defense. The authors first investigate the nature of adversarial attacks by analyzing two key sources of uncertainty for an adversary:

  1. Architecture Uncertainty: How well do attacks crafted for one DL architecture transfer to others?
  2. Domain Uncertainty: How well do attacks crafted for time-domain features transfer to frequency-domain features, and vice-versa?

Based on their findings that attacks are not directly transferable across different DL architectures or signal domains, they propose the ADE. This defense architecture comprises an ensemble of diverse deep learning models. Specifically, it includes models trained on:

  • In-phase and Quadrature (IQ) time-domain signals: These capture the raw signal waveform.
  • Frequency-domain signals: These are derived using the Discrete Fourier Transform (DFT) of the IQ signals.

The ADE combines the predictions from multiple DL architectures (FCNN, CNN, RNN, CRNN) trained on both time and frequency domain representations. This diversity is key to its robustness. The final modulation classification is an aggregation of the outputs from all models in the ensemble.

Datasets

The authors evaluate their proposed methodology on two publicly available datasets:

  • GNU RadioML2016.10a (Dataset A): This dataset uses 128-length observation windows. For their experiments, they focus on CPFSK, GFSK, PAM4, and QPSK modulation schemes.
  • RadioML2018.01a (Dataset B): This dataset uses 1024-length observation windows, which are downsampled to 128-length for consistency. They focus on OOK, 8ASK, BPSK, and FM modulation schemes.

Key Insights and Ideas

  • Transferability is Limited: Adversarial attacks are not universally transferable. Attacks crafted for one DL architecture or one signal domain (time vs. frequency) do not easily transfer to different architectures or domains. This is a crucial insight that forms the basis of their defense.
  • Diversity is Key: By combining models trained on different signal representations (time and frequency) and employing various DL architectures, the ensemble becomes significantly more resilient. An attack that fools one model might be easily handled by another within the ensemble.
  • Domain-Specific Robustness Varies: Certain architectures, like CNNs and CRNNs, show a greater inherent resilience to adversarial attacks, particularly when trained on frequency-domain features.
  • Black Box Attacks are Realistic: The paper emphasizes that adversaries often operate with limited knowledge (black box scenarios), making the transferability of attacks a primary concern. Their defense directly addresses this realistic threat model.

Key Results

  • Substantial Performance Improvement: The proposed ADE defense demonstrates significant improvements in AMC classification accuracy compared to baseline defenses across various adversarial attacks and potencies.
  • Mitigation of Transferable Attacks: The ADE effectively mitigates transferable attacks in both architecture uncertainty and signal domain uncertainty environments. For instance, in some scenarios, the ADE achieved classification accuracies over 90% even at a 5 dB PNR (Perturbation-to-Noise Ratio), while single models dropped to accuracies as low as 22%.
  • Robustness Across Datasets: The ADE shows consistent performance improvements on both Dataset A and Dataset B, indicating its generalizability.
  • CNN and CRNN Superiority: Among individual DL architectures, CNNs and CRNNs consistently achieved high accuracy and faster convergence, especially when trained on frequency-domain features.

Limitations and Future Work

  • Dataset Dependency: While the general trends hold, the degree of mitigation for transferable attacks can vary between datasets, suggesting that dataset characteristics might influence attack effectiveness and defense performance.
  • Computational Overhead: Ensemble methods inherently introduce higher computational complexity compared to single models, although the paper highlights that their CNN-based approach offers a good balance between performance and efficiency.
  • Future Work: The authors suggest adapting their proposed method for AMC in image classification settings, indicating the broader applicability of their ensemble-based defense strategy. They also note the need to further investigate performance in low SNR regimes.

Summary updated

Abstract

Deep learning-based automatic modulation classification (AMC) models are susceptible to adversarial attacks. Such attacks inject specifically crafted wireless interference into transmitted signals to induce erroneous classification predictions. Furthermore, adversarial interference is transferable in black box environments, allowing an adversary to attack multiple deep learning models with a single perturbation crafted for a particular classification model. In this work, we propose a novel wireless receiver architecture to mitigate the effects of adversarial interference in various black box attack environments. We begin by evaluating the architecture uncertainty environment, where we show that adversarial attacks crafted to fool specific AMC DL architectures are not directly transferable to different DL architectures. Next, we consider the domain uncertainty environment, where we show that adversarial attacks crafted on time domain and frequency domain features to not directly transfer to the altering domain. Using these insights, we develop our Assorted Deep Ensemble (ADE) defense, which is an ensemble of deep learning architectures trained on time and frequency domain representations of received signals. Through evaluation on two wireless signal datasets under different sources of uncertainty, we demonstrate that our ADE obtains substantial improvements in AMC classification performance compared with baseline defenses across different adversarial attacks and potencies.
BibTeX
@article{sahay2022deep,
  author = {Rajeev Sahay and Christopher G. Brinton and David J. Love},
  title = {{A Deep Ensemble-Based Wireless Receiver Architecture for Mitigating Adversarial Attacks in Automatic Modulation Classification}},
  journal = {IEEE Trans. Cogn. Commun. Netw.},
  year = {2022},
  url = {https://openreview.net/forum?id=Bjk0XEND0r},
}
Figure