UAV-Assisted Communication in Remote Disaster Areas using Imitation Learning

Alireza Shamsoshoara, Fatemeh Afghah, Erik Blasch, Jonathan D. Ashdown, Mehdi Bennis

Abstract: The damage to cellular towers during natural and man-made disasters can disturb the communication services for cellular users. One solution to the problem is using unmanned aerial vehicles to augment the desired communication network. The paper demonstrates the design of a UAV-Assisted Imitation Learning (UnVAIL) communication system that relays the cellular users' information to a neighbor base station. Since the user equipment (UEs) are equipped with buffers with limited capacity to hold packets, UnVAIL alternates between different UEs to reduce the chance of buffer overflow, positions itself optimally close to the selected UE to reduce service time, and uncovers a network pathway by acting as a relay node. UnVAIL utilizes Imitation Learning (IL) as a data-driven behavioral cloning approach to accomplish an optimal scheduling solution. Results demonstrate that UnVAIL performs similar to a human expert knowledge-based planning in communication timeliness, position accuracy, and energy consumption with an accuracy of 97.52% when evaluated on a developed simulator to train the UAV.

Introduction: Restoring Connectivity in Disaster Zones

When natural disasters like wildfires or floods strike, cellular base stations are often the first infrastructure to fail. This leaves local users (User Equipment, or UEs) isolated. Unmanned Aerial Vehicles (UAVs) offer a promising solution as mobile, line-of-sight relays. However, managing a UAV relay involves complex real-time optimization: the drone must decide which user to service to prevent buffer overflows while managing its own flight path and battery life.

System architecture of UnVAIL showing a UAV relaying data from ground users to a functional base station when the local tower is damaged.

Traditional Reinforcement Learning (RL) often requires extensive training time in the field, which is impractical during emergencies. Conversely, standard model-based optimization can be too computationally intensive for the limited onboard hardware of a drone. The researchers seek a "middle way" that is both fast and smart.

The UnVAIL Methodology

To address these challenges, the researchers developed UnVAIL (UAV-Assisted Imitation Learning). This system employs Behavioral Cloning (BC), a subset of Imitation Learning, to create a lightweight yet intelligent scheduling policy.

The Behavioral Cloning process where a UAV learner policy is trained to mimic an expert policy using a demonstration set.

The training process involves two main components:

  1. The Expert: A high-performance simulator that uses a complex utility function, Energy/Delay Throughput (EDT), to make optimal scheduling decisions based on user queue lengths and distances.
  2. The Learner: A Deep Neural Network (DNN) with three hidden layers (80, 160, and 80 neurons). It is trained to observe the system state—specifically the Buffer Status Report (BSR) of ground users—and predict the expert's next action.

The resulting model outputs two primary commands: selecting one of the five users to service and determining the UAV's movement (clockwise, counter-clockwise, or hover) along a circular path above the disaster zone.

Experimental Setup and Dataset

The system was evaluated using a custom-built UAV-assisted communication simulator in Python. The dataset comprised 28 hours of expert trajectories, covering the transmission of 550,000 packets. Features included queue lengths for five UEs, UAV coordinates, and battery levels. The data was split into 80% for training and 20% for validation, with a separate test set for final evaluation on unseen scenarios.

Key Results and Insights

The UnVAIL system demonstrated high fidelity to the expert's decision-making process:

  • High Accuracy: The model achieved 97.52% accuracy in mimicking expert user selection on unseen test data.
  • Edge-Ready Performance: Decision-making latency was reduced to less than 1ms on an NVIDIA Jetson Nano, proving the system's viability for real-world autonomous hardware.
  • Implicit Optimization: The model successfully learned to balance the trade-off between moving closer to a user (to speed up data transfer) and minimizing mobility energy consumption.

Limitations and Future Directions

While highly effective, the model is sensitive to "distribution shift." When packet arrival rates were altered from the training data, accuracy dropped to approximately 79%. Future work aims to improve robustness using techniques like DAgger (Dataset Aggregation) or Inverse Reinforcement Learning (IRL), which would allow the UAV to understand the underlying goals of the expert rather than just copying actions.

Summary updated

Abstract

The damage to cellular towers during natural and man-made disasters can disturb the communication services for cellular users. One solution to the problem is using unmanned aerial vehicles to augment the desired communication network. The paper demonstrates the design of a UAV-Assisted Imitation Learning (UnVAIL) communication system that relays the cellular users' information to a neighbor base station. Since the user equipment (UEs) are equipped with buffers with limited capacity to hold packets, UnVAIL alternates between different UEs to reduce the chance of buffer overflow, positions itself optimally close to the selected UE to reduce service time, and uncovers a network pathway by acting as a relay node. UnVAIL utilizes Imitation Learning (IL) as a data-driven behavioral cloning approach to accomplish an optimal scheduling solution. Results demonstrate that UnVAIL performs similar to a human expert knowledge-based planning in communication timeliness, position accuracy, and energy consumption with an accuracy of 97.52% when evaluated on a developed simulator to train the UAV.
BibTeX
@article{shamsoshoara2021uav,
  author = {Alireza Shamsoshoara and Fatemeh Afghah and Erik Blasch and Jonathan D. Ashdown and Mehdi Bennis},
  title = {{UAV-Assisted Communication in Remote Disaster Areas using Imitation Learning}},
  journal = {IEEE Open J. Commun. Soc.},
  year = {2021},
  doi = {10.1109/ojcoms.2021.3067001},
}
Figure