Intelligent Reflecting Surface Assisted Anti-Jamming Communications Based on Reinforcement Learning

Helin Yang, Zehui Xiong, Jun Zhao, Dusit Niyato, Qingqing Wu, Massimo Tornatore, Stefano Secci

Abstract

Malicious jamming launched by smart jammers, which attack legitimate transmissions, has been regarded as one of the critical security challenges in wireless communications. This paper exploits intelligent reflecting surfaces (IRS) to enhance anti-jamming communication performance and mitigate jamming interference by adjusting the surface reflecting elements. Aiming to enhance performance against smart jammers, an optimization problem for jointly optimizing power allocation at the base station (BS) and reflecting beamforming at the IRS is formulated. As jamming models and behaviors are dynamic and unknown, a Win or Learn Fast Policy Hill-Climbing (WoLF-PHC) learning approach is proposed to jointly optimize the anti-jamming strategy without prior knowledge of the jammer. Simulation results demonstrate that the proposed approach efficiently improves both the IRS-assisted system rate and transmission protection level compared with existing solutions.

Introduction: The Challenge of Smart Jamming

Wireless communication is inherently vulnerable to jamming, where an adversary transmits noise to drown out legitimate signals. Traditional defenses, such as frequency hopping or active relays, often incur high hardware costs, excessive power consumption, or increased architectural complexity. The challenge is amplified by "smart" jammers that adapt their power and strategy dynamically.

In such environments, traditional optimization techniques often fail because the objective function changes constantly as the jammer reacts. This paper addresses the non-convex optimization problem of maximizing legitimate users' data rates while minimizing base station (BS) transmit power, navigating a landscape where power allocation and IRS phase shifts are intricately coupled.

Proposed Methodology

The core innovation of this research lies in the synergy between Intelligent Reflecting Surfaces (IRS) and a specialized Reinforcement Learning (RL) framework.

1. Intelligent Reflecting Surface (IRS)

An IRS is a passive surface composed of numerous low-cost elements. Each element can independently shift the phase of an incoming electromagnetic wave. By carefully tuning these reflections, the IRS can strengthen the desired signal at the user and potentially cancel out or bypass the jamming signal. Unlike traditional relays, the IRS is nearly passive, providing a new degree of freedom to reshape the wireless environment itself.

2. The WoLF-PHC Algorithm

To handle the dynamic nature of the jammer, the authors employ the Win or Learn Fast Policy Hill-Climbing (WoLF-PHC) algorithm. Standard Q-Learning can be slow to adapt or may converge to sub-optimal solutions in adversarial settings. WoLF-PHC introduces a variable learning rate:

  • "Win" (Cautious): When the agent performs better than its average policy, it learns slowly to maintain stability.
  • "Learn Fast" (Aggressive): When the agent performs worse than average, it adapts its policy quickly to escape the poor strategy.
The proposed reinforcement learning framework for joint optimization of power allocation and reflecting beamforming.

3. Joint Optimization

The RL agent, located at the BS, treats the environment as a state-space comprising previous jamming power, Signal-to-Interference-plus-Noise Ratio (SINR), and channel coefficients. It takes actions that simultaneously decide the transmit power for each user and the phase shift for every IRS element, effectively managing both external jamming and Inter-User Interference (IUI).

Experimental Setup

The authors evaluated the system using simulated environments based on quasi-static flat-fading channel models. The setup involved a 100m x 100m area containing a BS, an IRS (with elements $M$ ranging from 20 to 100), and a multi-antenna jammer. Obstacles were simulated to block the direct path between the BS and users, highlighting the IRS's critical role in creating alternative signal paths.

Key Results and Discussion

The simulation results highlight several critical advantages of the proposed IRS-assisted RL approach:

  • Superior System Rate: The performance improves almost linearly with the number of IRS elements ($M$). With $M=100$, the proposed system achieved a rate gain of approximately 15.47 bits/s/Hz compared to a system without IRS assistance.
  • Faster Convergence: The WoLF-PHC approach reached a stable, high-performing policy significantly faster than standard "Fast Q-Learning" and far outperformed simple greedy strategies.
  • Robustness to Adversaries: Even when the jammer's power was unknown and varied between 15 and 40 dBm, the RL agent successfully learned to maintain a high SINR.
System rate performance comparison against the number of IRS elements (M) for different algorithms.

Conclusion and Future Work

This research demonstrates that IRS is a powerful tool for enhancing the security and reliability of wireless networks against malicious jamming. By using the WoLF-PHC algorithm, the system can "out-engineer" smart jammers without requiring a mathematical model of the jammer's logic.

Future research directions include addressing the challenge of obtaining accurate Channel State Information (CSI) in the presence of interference and managing the "curse of dimensionality" as the number of IRS elements grows. Additionally, the authors suggest extending this framework to Visible Light Communications (VLC), where IRS-like mirrors could steer light beams around physical obstacles and jammers.

Summary updated

BibTeX
@inproceedings{yang2020intelligent,
  author = {Helin Yang and Zehui Xiong and Jun Zhao and Dusit Niyato and Qingqing Wu and Massimo Tornatore and Stefano Secci},
  title = {{Intelligent Reflecting Surface Assisted Anti-Jamming Communications Based on Reinforcement Learning}},
  booktitle = {GLOBECOM},
  year = {2020},
  url = {https://openreview.net/forum?id=cAQjutzS5B},
}
Figure