Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems using DRL and PSO

Marwan Dhuheir, Aiman Erbad, Ala I. Al-Fuqaha, Mohsen Guizani

Abstract

Unmanned Aerial Vehicles (UAVs) are increasingly utilized as aerial base stations to provide wireless services in dynamic environments such as large-scale events or disaster recovery zones. However, their deployment is often hindered by limited energy budgets and signal blockages in urban landscapes. This paper presents a joint optimization framework that coordinates a UAV swarm and multiple Reconfigurable Intelligent Surfaces (RIS) to maximize throughput and coverage. By decoupling the problem into a two-step solution—using Deep Reinforcement Learning (DRL) for path planning and Particle Swarm Optimization (PSO) for RIS phase configuration—the system achieves significant improvements in Quality of Service (QoS). Simulation results indicate that this approach outperforms baseline mobility models by 30% and brute-force methods by 20%.

Introduction

The versatility and maneuverability of UAVs make them ideal candidates for providing temporary wireless coverage. Despite their potential, two primary challenges persist: the inherent resource constraints of the aircraft and the high probability of signal blockage in dense urban environments. While Reconfigurable Intelligent Surfaces (RIS) offer a passive, energy-efficient way to "steer" radio waves around obstacles, managing a multi-UAV, multi-RIS network introduces a complex, non-convex optimization problem that is difficult to solve using traditional linear programming.

This research addresses these complexities by proposing a hybrid optimization model. The goal is to ensure high-quality communication for ground users with diverse service requirements while maintaining the operational safety and efficiency of the UAV swarm.

Methodology

The proposed system utilizes a decoupled optimization strategy to manage the high-dimensional state space of a multi-UAV, multi-RIS environment.

UAV Path Planning via Deep Reinforcement Learning (DRL)

To navigate the environment effectively, the system employs DRL agents (testing both Deep Q-Networks and Actor-Critic architectures). The agents are tasked with maximizing user coverage while adhering to battery constraints and avoiding mid-air collisions. The DRL framework allows the UAVs to learn optimal trajectories by interacting with the environment, receiving rewards for serving users and penalties for safety violations.

RIS Configuration via Particle Swarm Optimization (PSO)

Once the UAV positions are established by the DRL agent, the system must optimize the phase shifts of the RIS elements to maximize data throughput. Given the enormous search space of possible phase configurations, the authors utilize Particle Swarm Optimization (PSO). This metaheuristic approach, inspired by the collective behavior of bird flocks, iteratively refines the RIS settings to find a near-optimal configuration that enhances the signal-to-noise ratio (SNR) for ground users.

Heterogeneous QoS Awareness

Unlike uniform models, this framework accounts for varying service types. It enforces specific SNR thresholds to ensure quality for different applications:

  • Video: 30 dB
  • Data: 25 dB
  • Audio: 20 dB

Experimental Setup

The model was validated in a custom-built simulation environment representing a 400m x 400m urban area. To reflect realistic demand, 400 ground users were distributed according to a power-law distribution, creating "hotspots" similar to those found in stadiums or festival stages. The channel modeling incorporated probabilistic Line-of-Sight (LoS) for direct UAV-to-user links and Rician fading channels for indirect links involving RIS reflections.

Key Results and Discussion

The synergy between DRL and PSO proved highly effective for real-time decision-making. Key findings include:

  • Superior Performance: The DRL-PSO approach yielded a 30% improvement in QoS satisfaction compared to standard Random Waypoint mobility models. Notably, it also outperformed a brute-force search by 20%, suggesting that the hybrid approach finds more efficient global solutions than exhaustive local searches.
  • Scalability: The system demonstrated strong scaling properties; as the number of RIS elements ($M$) increased, the throughput gains grew significantly, with the DRL-PSO method consistently maintaining the lead over competitive baselines.
  • Intelligent Navigation: The DRL agents successfully learned to prioritize high-density user clusters (the "hotspots"), effectively "hunting" for areas with the highest potential reward while maintaining swarm safety.

Conclusion and Future Work

This work demonstrates that combining DRL for high-level navigation and PSO for fine-grained hardware configuration provides a robust solution for complex aerial communication networks. The use of passive RIS elements significantly enhances coverage without the energy overhead of active relays.

Future research will focus on addressing inter-cell interference in high-density swarms and mitigating the Doppler effects caused by high-speed UAV mobility. Additionally, transitioning from a centralized coordination model to a fully decentralized swarm intelligence architecture remains a primary objective for enhancing system resilience.

Summary updated

Abstract

Recently, Unmanned Aerial Vehicles (UAVs) have attracted the attention of researchers in academia and industry for providing wireless services to ground users in diverse scenarios like festivals, large sporting events, natural and man-made disasters due to their advantages in terms of versatility and maneuverability. However, the limited resources of UAVs (e.g., energy budget and different service requirements) can pose challenges for adopting UAVs for such applications. Our system model considers a UAV swarm that navigates an area, providing wireless communication to ground users with RIS support to improve the coverage of the UAVs. In this work, we introduce an optimization model with the aim of maximizing the throughput and UAVs coverage through optimal path planning of UAVs and multi-RIS phase configurations. The formulated optimization is challenging to solve using standard linear programming techniques, limiting its applicability in real-time decision-making. Therefore, we introduce a two-step solution using deep reinforcement learning and particle swarm optimization. We conduct extensive simulations and compare our approach to two competitive solutions presented in the recent literature. Our simulation results demonstrate that our adopted approach is 20 \% better than the brute-force approach and 30\% better than the baseline solution in terms of QoS.
BibTeX
@inproceedings{dhuheir2024multi,
  author = {Marwan Dhuheir and Aiman Erbad and Ala I. Al-Fuqaha and Mohsen Guizani},
  title = {{Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems using DRL and PSO}},
  booktitle = {ICC},
  year = {2024},
  url = {https://openreview.net/forum?id=FHQcyL06ep},
}
Figure