Trans-UTPA: PSO and MADDPG based multi-UAVs trajectory planning algorithm for emergency communication

Jie Li, Shuang Cao, Xianjie Liu, Ruiyun Yu, Xingwei WangFrontiers Neurorobotics 2022 · DOI

Abstract

In the wake of natural disasters, restoring communication is a critical priority. Unmanned Aerial Vehicles (UAVs) serve as effective "flying base stations," yet their deployment is hindered by limited battery life and the complexity of 3D trajectory planning. This paper introduces Trans-UTPA, a hybrid algorithm that integrates swarm intelligence with Transformer-based reinforcement learning. By optimizing flight paths for energy efficiency and mission priority, Trans-UTPA achieves a 60% reduction in mid-air collisions and a 13% improvement in mission rewards compared to existing state-of-the-art methods.

Introduction

When natural disasters strike, ground-based communication infrastructure often fails. UAVs are ideal for restoring connectivity, but they face a grueling constraint: extremely limited battery life. Planning trajectories for a swarm of drones requires solving a multi-dimensional puzzle: identifying high-density user areas, prioritizing mission-critical tasks, and navigating 3D environments while avoiding obstacles. Existing algorithms often struggle with action space limitations or fail to scale as the number of drones increases.

Methodology

The Trans-UTPA pipeline consists of three distinct stages designed to optimize swarm behavior from initial target identification to real-time navigation.

1. Clustering and Task Prioritization

The first stage uses Particle Swarm Optimization (PSO) to group scattered individuals on the ground into "Target Points of Interest" (target-POIs). This ensures drones focus on high-density areas. Subsequently, a Neural Collaborative Filtering (NCF) algorithm ranks these POIs, helping the swarm decide which areas require immediate networking services versus simple data harvesting.

Flowchart of the PSO-based clustering and NCF recommendation process.

2. Transformer-Based Path Planning

The core innovation lies in the path-planning stage, which utilizes a Multi-Agent Proximal Policy Optimization (MAPPO) framework. Unlike traditional approaches that use Recurrent Neural Networks (RNNs), Trans-UTPA employs a Transformer architecture.

Transformers use "self-attention" to process state information in parallel rather than sequentially. This allows the model to better understand the complex relationships between multiple drones and their environment simultaneously, leading to superior generalization and faster learning. The model also incorporates a detailed Green Energy Consumption Model that accounts for the energy costs of horizontal flight, vertical movement, hovering, and fuselage rotation.

The multi-agent actor structure within the Transformer-based framework, showing the processing of observations and hidden states into actions.

Experimental Setup and Datasets

To evaluate the algorithm in realistic scenarios, the authors utilized:

  • Mafengwo Tourism Dataset: 7,366 scenic spot locations in Liaoning, China, providing a non-uniform spatial distribution to test PSO clustering.
  • Synthetic Grid Environments: Simulated 32x32 and 64x64 maps with varying obstacle densities (0.1 to 0.3) to evaluate collision avoidance and mission success rates.

Key Results and Discussion

Trans-UTPA was compared against the USCTP algorithm and standard heuristic methods like A*. Key findings include:

  • Enhanced Safety: Mid-air collisions were reduced by 60% compared to USCTP.
  • Operational Efficiency: The average mission reward—a composite metric of task completion and energy savings—increased by 13%.
  • Superior Coverage: In a 10-minute window, Trans-UTPA enabled a single UAV to visit 25 POIs, a significant improvement over the 6 POIs managed by the A* algorithm.
  • Robustness: The algorithm maintained a high success rate even as obstacle density increased, demonstrating its effectiveness in cluttered disaster environments.

Conclusion

Trans-UTPA represents a significant step forward in multi-UAV coordination for emergency response. By decoupling policy learning from rigid action spaces via Transformers and implementing strategic task allocation, the system maximizes the utility of the entire swarm. Future work will focus on transitioning from 3D grid simulations to high-fidelity environments that account for real-world aerodynamics and wind gusts, as well as improving the computational scalability for even larger drone swarms.

Summary updated

BibTeX
@article{li2022trans,
  author = {Jie Li and Shuang Cao and Xianjie Liu and Ruiyun Yu and Xingwei Wang},
  title = {{Trans-UTPA: PSO and MADDPG based multi-UAVs trajectory planning algorithm for emergency communication}},
  journal = {Frontiers Neurorobotics},
  year = {2022},
  doi = {10.3389/fnbot.2022.1076338},
}
Figure