Abstract
This paper investigates the problem of anti-jamming communication in dynamic and intelligent jamming environments. A sequential deep reinforcement learning algorithm (SDRLA) without prior information is proposed, using raw spectrum information as input. The SDRLA framework consists of two primary components: first, a deep learning module utilizing the sliding window principle to identify jamming patterns; and second, a reinforcement learning module for online channel selection based on those recognized patterns. Additionally, a channel switching cost is introduced to balance the trade-off between throughput and operational overhead. By leveraging both deep learning and reinforcement learning, this method achieves rapid and effective anti-jamming channel selection without the need for modeling the jammer's characteristics. Simulation results demonstrate the convergence and effectiveness of the proposed SDRLA algorithm, showing superior performance compared to single-mode reinforcement learning.
Introduction
Wireless communication faces persistent threats from malicious jamming. While traditional defenses like Frequency Hopping Spread Spectrum (FHSS) are standard, modern "intelligent" jammers can adapt their strategies to disrupt links more effectively. Existing machine learning solutions often suffer from slow adaptation—taking significant time to "re-learn" when a jammer switches tactics—and information loss, as they often simplify complex spectrum data into basic features, discarding the subtle "fingerprints" of specific jamming attacks.
This research proposes a system capable of real-time environment sensing and pattern identification to switch to optimal channels without prior knowledge of the jammer’s hardware or intent.
The SDRLA Framework
The researchers proposed a two-stage architecture called the Sequential Deep Reinforcement Learning Algorithm (SDRLA).
1. The Perception Stage (CNN)
Instead of relying on simplified metrics, the system collects raw spectrum information to generate "spectral waterfall diagrams" (thermal charts showing frequency vs. time). A Convolutional Neural Network (CNN) is trained to classify these images into specific jamming categories, such as Sweep Jamming, Comb Jamming, or Double Sweep.
Novelty: The Jamming Transition Band A key innovation is the handling of "jamming transition bands." When a jammer switches modes, the spectrum becomes chaotic. By labeling these transition periods as distinct classes, the authors prevent the CNN from making erratic decisions during the switch, smoothing the system's behavior.
2. The Decision Stage (Q-Learning)
Once the CNN identifies the pattern, the system triggers a specific Q-learning agent optimized for that environment. This sequential approach ensures the Reinforcement Learning (RL) agent only needs to master the best moves for the current, identified threat, rather than the entire universe of possibilities.
3. Channel Switching Cost
To mirror real-world hardware constraints, the authors integrated a channel switching cost into the reward function. Since frequency hopping creates overhead, the algorithm is incentivized to remain on a "good enough" channel rather than constantly chasing a perfect one if the cost of moving is prohibitively high.
Experimental Setup and Results
The study utilized simulated datasets mimicking electronic warfare scenarios, featuring various jamming modes and 200x200 pixel thermal charts representing 200ms of spectrum data. Key findings include:
- Throughput Stability: In dynamic scenarios with random pattern switching, SDRLA maintained a cumulative throughput near maximum levels, whereas single-mode RL fluctuated significantly.
- Convergence Speed: SDRLA converged faster than standard RL because it can invoke a pre-fitted Q-table immediately after the CNN identifies a pattern shift.
- Overhead Reduction: The switching cost factor reduced unnecessary channel hops by 50% to 70%, lowering computational and signaling overhead without sacrificing communication quality.
Conclusion and Future Work
While SDRLA shows great promise, it currently depends on supervised learning and labeled datasets for the CNN. Future work may address how the system handles entirely novel jamming patterns. Additionally, while the paper assumes a reliable control link for coordination, protecting this link in high-interference environments remains a challenge. Finally, implementing such computationally intensive CNNs on low-power radio edge devices is a hurdle for practical deployment.
Summary updated