Pattern-Aware Intelligent Anti-Jamming Communication: A Sequential Deep Reinforcement Learning Approach

Songyi Liu, Yifan Xu, Xueqiang Chen, Ximing Wang, Meng Wang, Wen Li, Yangyang Li, Yuhua XuIEEE Access 2019 · OpenReview

Abstract

This paper investigates the problem of anti-jamming communication in dynamic and intelligent jamming environments. A sequential deep reinforcement learning algorithm (SDRLA) without prior information is proposed, using raw spectrum information as input. The SDRLA framework consists of two primary components: first, a deep learning module utilizing the sliding window principle to identify jamming patterns; and second, a reinforcement learning module for online channel selection based on those recognized patterns. Additionally, a channel switching cost is introduced to balance the trade-off between throughput and operational overhead. By leveraging both deep learning and reinforcement learning, this method achieves rapid and effective anti-jamming channel selection without the need for modeling the jammer's characteristics. Simulation results demonstrate the convergence and effectiveness of the proposed SDRLA algorithm, showing superior performance compared to single-mode reinforcement learning.

Introduction

Wireless communication faces persistent threats from malicious jamming. While traditional defenses like Frequency Hopping Spread Spectrum (FHSS) are standard, modern "intelligent" jammers can adapt their strategies to disrupt links more effectively. Existing machine learning solutions often suffer from slow adaptation—taking significant time to "re-learn" when a jammer switches tactics—and information loss, as they often simplify complex spectrum data into basic features, discarding the subtle "fingerprints" of specific jamming attacks.

This research proposes a system capable of real-time environment sensing and pattern identification to switch to optimal channels without prior knowledge of the jammer’s hardware or intent.

The SDRLA Framework

The researchers proposed a two-stage architecture called the Sequential Deep Reinforcement Learning Algorithm (SDRLA).

1. The Perception Stage (CNN)

Instead of relying on simplified metrics, the system collects raw spectrum information to generate "spectral waterfall diagrams" (thermal charts showing frequency vs. time). A Convolutional Neural Network (CNN) is trained to classify these images into specific jamming categories, such as Sweep Jamming, Comb Jamming, or Double Sweep.

Spectral waterfall diagrams showing the distinction between user signals and malicious jamming patterns.

Novelty: The Jamming Transition Band A key innovation is the handling of "jamming transition bands." When a jammer switches modes, the spectrum becomes chaotic. By labeling these transition periods as distinct classes, the authors prevent the CNN from making erratic decisions during the switch, smoothing the system's behavior.

Visualization of the jamming pattern transition, highlighting the period where the interference strategy shifts and the user signal is jammed.

2. The Decision Stage (Q-Learning)

Once the CNN identifies the pattern, the system triggers a specific Q-learning agent optimized for that environment. This sequential approach ensures the Reinforcement Learning (RL) agent only needs to master the best moves for the current, identified threat, rather than the entire universe of possibilities.

3. Channel Switching Cost

To mirror real-world hardware constraints, the authors integrated a channel switching cost into the reward function. Since frequency hopping creates overhead, the algorithm is incentivized to remain on a "good enough" channel rather than constantly chasing a perfect one if the cost of moving is prohibitively high.

Experimental Setup and Results

The study utilized simulated datasets mimicking electronic warfare scenarios, featuring various jamming modes and 200x200 pixel thermal charts representing 200ms of spectrum data. Key findings include:

  • Throughput Stability: In dynamic scenarios with random pattern switching, SDRLA maintained a cumulative throughput near maximum levels, whereas single-mode RL fluctuated significantly.
  • Convergence Speed: SDRLA converged faster than standard RL because it can invoke a pre-fitted Q-table immediately after the CNN identifies a pattern shift.
  • Overhead Reduction: The switching cost factor reduced unnecessary channel hops by 50% to 70%, lowering computational and signaling overhead without sacrificing communication quality.

Conclusion and Future Work

While SDRLA shows great promise, it currently depends on supervised learning and labeled datasets for the CNN. Future work may address how the system handles entirely novel jamming patterns. Additionally, while the paper assumes a reliable control link for coordination, protecting this link in high-interference environments remains a challenge. Finally, implementing such computationally intensive CNNs on low-power radio edge devices is a hurdle for practical deployment.

Summary updated

Abstract

This paper investigates the problem of anti-jamming communication in dynamic and intelligent jamming environment. A sequential deep reinforcement learning algorithm (SDRLA) without prior information is proposed, and raw spectrum information is used as the input of SDRLA. The proposed SDRLA algorithm mainly contains two parts: Firstly, deep learning and sliding window principle are introduced to identify jamming patterns; Secondly, reinforcement learning is carried out to make on-line channel selection based on recognized jamming patterns. In addition, channel switching cost is introduced for the purpose of formulating the trade-off relationship between throughput and overhead. Taking advantage of both deep learning and reinforcement learning, this method can realize rapid and effective anti-jamming channel selection with no need for modeling the jammer's characteristics. Simulation results show the convergence and effectiveness of the proposed SDRLA algorithm. Compared with single-mode reinforcement learning, our approach can reach better convergence performance and higher utility.
BibTeX
@inproceedings{liu2019pattern,
  author = {Songyi Liu and Yifan Xu and Xueqiang Chen and Ximing Wang and Meng Wang and Wen Li and Yangyang Li and Yuhua Xu},
  title = {{Pattern-Aware Intelligent Anti-Jamming Communication: A Sequential Deep Reinforcement Learning Approach}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=NmbIRF1AHF},
}
Figure