Abstract
This paper introduces SAC-TLS, a Deep Reinforcement Learning (DRL) framework designed to optimize data freshness in remote IoT networks. By leveraging multiple Autonomous Aerial Vehicles (AAVs) to form a Virtual Antenna Array (VAA) through distributed beamforming, the system minimizes the Age of Information (AoI) while balancing energy consumption and trajectory planning. The proposed method significantly outperforms standard DRL baselines in both data freshness and convergence speed.
Introduction
In remote monitoring scenarios such as forest fire detection or border surveillance, ground-based sensor networks (SNs) often lack the transmission power to reach distant base stations (BS). Autonomous Aerial Vehicles (AAVs) are frequently deployed as mobile relays to bridge this gap. However, traditional relaying methods often suffer from high data staleness, or Age of Information (AoI), due to the "propulsion-repositioning cycle"—where drones must fly back and forth between sensors and the base station to upload data.
System Model and Distributed Beamforming
The researchers propose a novel architecture where a fleet of AAVs coordinates to perform distributed beamforming (DB). By synchronizing their signals, the drones act as a single high-gain Virtual Antenna Array (VAA). This technological leap allows the drones to transmit data over much longer distances, reducing the need for frequent return flights to the base station and allowing them to stay closer to the sensors for longer durations.
The SAC-TLS Framework
To manage the complex coordination of trajectories and communication schedules, the authors developed SAC-TLS, an enhanced version of the Soft Actor-Critic algorithm. The framework incorporates three critical architectural components:
- Temporal Sequence Input (T): Processes a window of past environmental states to anticipate sensor data patterns.
- Layer Normalization Gated Recurrent Unit (L): Integrates LNGRU cells to stabilize gradient flow and capture long-term dependencies in drone missions.
- Squeeze-and-Excitation Block (S): Implements a channel attention mechanism that helps the network prioritize sensors with the highest (oldest) AoI.
Additionally, the system utilizes Dynamic Proximity-based Action Mapping (DPAM) to streamline the learning process by linking communication decisions directly to the drones' physical proximity to sensors.
Comparative Analysis and Novelty
The proposed framework is uniquely comprehensive compared to existing literature. While previous studies often focused on single-AAV systems or ignored energy constraints, this work jointly optimizes multi-AAV trajectories, communication scheduling, and energy efficiency within a DRL framework that accounts for temporal dependencies.
Key Results and Insights
Experimental results in a simulated 400m x 400m environment with 60 sensor nodes demonstrate the efficacy of SAC-TLS:
- AoI Reduction: The algorithm reduced the average Age of Information by 17.3% compared to standard SAC.
- Efficiency and Convergence: SAC-TLS achieved 24.5% higher cumulative rewards and converged 1.6x faster than baseline methods like PPO and TD3.
- Deployment Readiness: Model pruning tests showed that the neural network could be reduced by 90% while maintaining 80% of its performance, indicating its suitability for resource-constrained onboard hardware.
Conclusion
By integrating distributed beamforming with advanced attention-based DRL, this research provides a scalable solution for maintaining data freshness in remote IoT monitoring. Future work will aim to address the challenges of nanosecond-level phase synchronization in turbulent real-world conditions and explore 3D trajectory optimization.
Summary updated