Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems Using DRL and PSO

Marwan Dhuheir, Aiman Erbad, Ala I. Al-Fuqaha, Mohsen Guizani

TL;DR

This paper introduces a joint optimization framework that coordinates a swarm of Unmanned Aerial Vehicles (UAVs) and multiple Reconfigurable Intelligent Surfaces (RIS) to provide high-quality wireless coverage to ground users. By combining Deep Reinforcement Learning (DRL) for path planning and Particle Swarm Optimization (PSO) for RIS configuration, the system achieves up to a 30% improvement in Quality of Service (QoS) over standard baseline mobility models.

What Problem Does This Paper Address?

Deploying UAVs as "flying base stations" is an ideal solution for providing temporary wireless coverage during festivals, sporting events, or disaster recovery. However, two major hurdles remain:

  1. Resource Constraints: UAVs have limited battery life and must navigate complex environments while satisfying diverse user needs (e.g., high-bandwidth video vs. low-bandwidth audio).
  2. Signal Blockage: In crowded urban areas, line-of-sight signals are often blocked by buildings.

While Reconfigurable Intelligent Surfaces (RIS)—passive panels that reflect and "steer" radio waves—can mitigate blockages, managing a swarm of moving UAVs alongside multiple RIS units creates a massive, non-convex optimization problem. Most existing research focuses on single-UAV or single-RIS setups; this paper tackles the complexity of a multi-UAV, multi-RIS system.

Methods

The authors propose a decoupled, two-step optimization approach to handle the complexity of the system:

  • UAV Path Planning via DRL: To maximize user coverage while respecting battery limits and avoiding mid-air collisions, the authors employ Deep Reinforcement Learning. They tested both Deep Q-Networks (DQN) and Actor-Critic architectures. The DRL agent learns an optimal policy by interacting with the environment, receiving rewards for maximizing the number of served users within safety constraints.
  • RIS Configuration via PSO: Once the UAV positions are determined, the system must configure the phase shifts of the RIS elements to maximize data throughput. Because the search space for phase shifts is enormous, the authors use Particle Swarm Optimization (PSO)—a metaheuristic inspired by bird flocking—to iteratively find the best configuration.
  • Heterogeneous QoS Awareness: The model specifically accounts for different service types, setting minimum Signal-to-Noise Ratio (SNR) thresholds for video (30 dB), data (25 dB), and audio (20 dB).

Datasets

The study relies on a custom-built simulation environment rather than a static external dataset:

  • Environment: A 400m x 400m urban area.
  • User Distribution: 400 ground users distributed according to a power-law distribution. This simulates realistic "hotspots" where users cluster (like a stadium or festival stage) rather than being spread uniformly.
  • Channel Models: The simulation incorporates both direct links (UAV-to-user) using probabilistic Line-of-Sight (LoS) models and indirect links (UAV-to-RIS-to-user) using Rician fading channels.

Key Insights and Ideas

  • Synergy of DRL and PSO: The "aha moment" is the realization that DRL is excellent for the sequential, high-level logic of navigation, while PSO is better suited for the high-dimensional, fine-grained task of tuning hundreds of RIS reflecting elements.
  • Clustered User Awareness: By using a power-law distribution for users, the authors demonstrate that their DRL agent successfully learns to prioritize "busy" cells, effectively "hunting" for the highest density of users to maximize the reward.
  • Passive Efficiency: The paper emphasizes that RIS is significantly more energy-efficient than traditional active relays because the reflecting elements are passive, making them a perfect companion for energy-constrained UAV swarms.

Key Results

  • QoS Improvement: The proposed DRL-PSO approach outperformed a standard baseline (Random Waypoint mobility) by 30% and even beat a brute-force search approach by 20% in terms of overall QoS satisfaction.
  • Scalability: As the number of RIS elements ($M$) increases, the throughput gains scale significantly. The DRL-PSO approach consistently maintained the highest throughput compared to competitive solutions.
  • Coverage Efficiency: The DRL-based path planning achieved coverage levels nearly identical to an exhaustive brute-force search but with a fraction of the computational overhead, making it more viable for real-time deployment.

Limitations and Future Work

The authors acknowledge a few areas for further exploration:

  • Interference Management: While the model handles multiple UAVs, the current work does not deeply explore the complex inter-cell interference that occurs when many UAVs transmit simultaneously in close proximity.
  • Doppler Effects: The high mobility of UAVs can cause frequency shifts (the Doppler effect), which could impact signal decoding; the authors plan to incorporate this into future iterations.
  • Centralization: The current model assumes a level of centralized coordination to run the DRL and PSO algorithms; moving toward a fully decentralized "swarm intelligence" model remains a challenge.

Summary updated

Abstract

Recently, Unmanned Aerial Vehicles (UAVs) have attracted the attention of researchers in academia and industry for providing wireless services to ground users in diverse scenarios like festivals, large sporting events, natural and man-made disasters due to their advantages in terms of versatility and maneuverability. However, the limited resources of UAV s (e.g., energy budget and different service requirements) can pose challenges for adopting UAV s for such applications. Our system model considers a UAV swarm that navigates an area, providing wireless communication to ground users with RIS support to improve the coverage of the UAV s. In this work, we introduce an optimization model with the aim of maximizing the throughput and UAVs coverage through optimal path planning of UAVs and multi-RIS phase configurations. The formulated optimization is challenging to solve using standard linear programming techniques, limiting its applicability in real-time decision-making. Therefore, we introduce a two-step solution using deep reinforcement learning and particle swarm optimization. We conduct extensive simulations and compare our approach to two competitive solutions presented in the recent literature. Our simulation results demonstrate that our adopted approach is 20 % better than the brute-force approach and 30% better than the baseline solution in terms of QoS.
BibTeX
@inproceedings{dhuheir2024multi,
  author = {Marwan Dhuheir and Aiman Erbad and Ala I. Al-Fuqaha and Mohsen Guizani},
  title = {{Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems Using DRL and PSO}},
  booktitle = {ICC},
  year = {2024},
  url = {https://openreview.net/forum?id=FHQcyL06ep},
}

Loading related content...

Figure