Traj-Q-GPSR: A Trajectory-Informed and Q-Learning Enhanced GPSR Protocol for Mission-Oriented FANETs

Mengjing Wu, Bo Jiang, Siji Chen, Hong Xu, Tao Pang, Mingke Gao, Fei XiaDrones 2025 · DOI

TL;DR

This paper introduces Traj-Q-GPSR, a novel routing protocol for drone swarms that leverages predicted flight paths and Q-learning to make smarter, more adaptive routing decisions. By looking ahead and learning from experience, Traj-Q-GPSR significantly improves packet delivery, reduces delays, and enhances overall network efficiency in dynamic environments.

What Problem Does This Paper Address?

Flying Ad Hoc Networks (FANETs), composed of drones, are increasingly used for tasks like disaster relief and monitoring. However, their high speed, constant movement, and limited energy make traditional routing protocols struggle. Existing methods often rely on outdated information, leading to dropped packets, long delays, and inefficient communication. Specifically, the Greedy Perimeter Stateless Routing (GPSR) protocol, while scalable, falters in these dynamic scenarios due to its reliance on immediate, single-hop neighbor information. This paper aims to overcome these limitations by creating a more intelligent and predictive routing system for drone swarms.

Methods

The core innovation of Traj-Q-GPSR lies in its integration of two key concepts:

  • Trajectory-Informed Neighbor Awareness: Instead of just knowing who is nearby now, Traj-Q-GPSR uses the drones' pre-planned mission trajectories to predict future positions. This allows each drone to build a "two-hop neighbor table" that anticipates who will be within communication range in the near future. This proactive approach helps avoid routing "voids" where no immediate path exists.
  • Q-Learning for Adaptive Routing: A reinforcement learning technique called Q-learning is employed to dynamically optimize the selection of the next hop. The Q-learning agent considers a rich state space, including two-hop neighbor availability, link stability, queue length, and node mobility patterns. A multi-objective reward function guides the learning process, balancing factors like transmission efficiency, energy consumption, and avoiding routing voids. The learning rate and discount factor are also dynamically adjusted based on network conditions to ensure faster adaptation and stable convergence.

Additionally, a delay-aware queuing model is used to predict optimal packet transmission times, further reducing buffering and retransmissions.

Datasets

The paper does not mention any specific datasets used for training. Instead, it relies on extensive simulations using the ns-3 network simulator and MATLAB R2022b for analysis and plotting. Mobility scenarios were generated using BonnMotion, a tool that produces node movement trace files based on various mobility models.

Key Insights and Ideas

  • Looking Ahead is Crucial: The most significant insight is that predicting future drone positions (trajectory awareness) dramatically improves routing decisions compared to reacting only to the current state. This allows the system to proactively find stable paths.
  • Q-Learning as a Dynamic Decision-Maker: Q-learning proves to be a powerful tool for adapting to the highly dynamic nature of drone swarms. Its ability to learn from interactions and adjust policies in real-time is key to overcoming the challenges of changing network topologies.
  • Two-Hop Awareness is Better Than One: Expanding neighbor awareness beyond immediate contacts to include two-hop neighbors, especially when informed by trajectory predictions, significantly reduces routing voids and improves path selection.
  • Adaptive Learning for Robustness: Dynamically adjusting Q-learning parameters (learning rate and discount factor) based on network conditions (Q-value fluctuations and neighbor change rates) is essential for achieving both rapid adaptation and long-term stability.

Key Results

The simulations, conducted across diverse mobility, density, and traffic conditions, show Traj-Q-GPSR consistently outperforming the traditional GPSR protocol:

  • Packet Loss: Up to 23% lower packet loss.
  • End-to-End Delay: Over 80% reduction in average end-to-end delay.
  • Routing Efficiency: Up to 52% improvement.
  • Throughput: Up to 37% improvement.

These improvements are attributed to the protocol's ability to mitigate congestion, avoid routing voids, and maintain stable links in dynamic environments.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that Traj-Q-GPSR incurs slightly higher communication overhead due to the exchange of trajectory information. However, this is shown to be amortized in denser topologies.
  • The current approach assumes predictable trajectory updates.
  • Future Work:
    • The authors aim to extend the framework to handle uncertain or asynchronous trajectory updates.
  • Further research will focus on improving learning convergence under constrained communication conditions.

Summary updated

Abstract

Routing in flying ad hoc networks (FANETs) is hindered by high mobility, trajectory-induced topology dynamics, and energy constraints. Conventional topology-based or position-based protocols often fail due to stale link information and limited neighbor awareness. This paper proposes a trajectory-informed routing protocol enhanced by Q-learning: Traj-Q-GPSR, tailored for mission-oriented UAV swarm networks. By leveraging mission-planned flight trajectories, the protocol builds time-aware two-hop neighbor tables, enabling routing decisions based on both current connectivity and predicted link availability. This spatiotemporal information is integrated into a reinforcement learning framework that dynamically optimizes next-hop selection based on link stability, queue length, and node mobility patterns. To further enhance adaptability, the learning parameters are adjusted in real time according to network dynamics. Additionally, a delay-aware queuing model is introduced to forecast optimal transmission timing, thereby reducing buffering overhead and mitigating redundant retransmissions. Extensive ns-3 simulations across diverse mobility, density, and CBR connections demonstrate that the proposed protocol consistently outperforms GPSR, achieving up to 23% lower packet loss, over 80% reduction in average end-to-end delay, and improvements of up to 37% and 52% in throughput and routing efficiency, respectively.
BibTeX
@inproceedings{wu2025traj,
  author = {Mengjing Wu and Bo Jiang and Siji Chen and Hong Xu and Tao Pang and Mingke Gao and Fei Xia},
  title = {{Traj-Q-GPSR: A Trajectory-Informed and Q-Learning Enhanced GPSR Protocol for Mission-Oriented FANETs}},
  booktitle = {Drones},
  year = {2025},
  doi = {10.3390/drones9070489},
}
Figure