Introduction
In the evolving landscape of the Internet of Things (IoT), applications such as environmental monitoring and industrial sensing increasingly depend on the "freshness" of data. Traditional performance metrics like throughput or latency often fail to capture the temporal relevance of information. To address this, researchers utilize the Age of Information (AoI), defined as the time elapsed since the most recent update was generated by a sensor.
Unmanned Aerial Vehicles (UAVs) offer a flexible solution for data collection, as their mobility allows them to establish high-quality communication links by flying in close proximity to sensors. However, UAV-assisted networks face significant operational hurdles, primarily limited flight time and finite battery energy. While previous studies have focused on maximizing throughput or minimizing flight duration, they often overlook the complex energy dynamics of rotary-wing aircraft. This paper proposes a joint optimization of UAV flight trajectories and sensor transmission scheduling to minimize network-wide AoI under strict energy constraints.
System Model and Methodology
The authors model the data collection task as a Markov Decision Process (MDP), aiming to find an optimal policy that balances data freshness against energy consumption.
The methodology incorporates several key components:
- Realistic Energy Model: A detailed propulsion power consumption model for rotary-wing UAVs is employed, distinguishing between the energy costs of hovering and flying at various speeds.
- State and Action Spaces: The state space tracks the UAV's grid coordinates, the current AoI of all sensors, remaining mission time, and remaining energy. In each time slot, the UAV chooses an action: moving in a cardinal direction (North, South, East, West), hovering, and selecting a sensor for data transmission.
- Reward Function: The agent is incentivized to minimize the weighted sum of AoI. To ensure mission safety, heavy penalties are applied if the UAV fails to reach the destination before exhausting its time or energy budget.
- Deep Q-Network (DQN): To handle the high-dimensional state space, a DQN architecture with experience replay and target networks is used to stabilize the learning process.
Experimental Setup
The proposed approach was evaluated in a simulated 500m × 500m environment divided into a 20 × 20 grid. Sensors were randomly distributed throughout the region. The simulation utilized realistic physical constants for air density, rotor speed, and channel gain to ensure the validity of the energy and communication models.
Results and Discussion
The DQN-based policy was compared against two heuristic baselines: an AoI-based algorithm (prioritizing the stalest sensor) and a Distance-based algorithm (a nearest-neighbor approach).
The results demonstrate that the DRL approach consistently achieves lower average AoI across various scenarios. Notably, as the number of sensors ($N$) increases, the performance gap between the DQN-based method and the baselines widens. This suggests that the DRL agent is significantly more effective at managing the complex scheduling conflicts that arise in denser IoT networks.
Furthermore, the study found that increasing the sensor coverage radius ($R$) leads to a decrease in AoI. A larger $R$ allows the UAV to collect data from multiple sensors simultaneously or from a greater distance, thereby reducing the total flight time required to refresh the network's data.
Conclusion and Future Work
This research highlights that flight trajectory and transmission scheduling are intrinsically linked; the drone's path determines communication opportunities, while sensor AoI requirements should drive the drone's movement. By incorporating energy constraints directly into the reinforcement learning reward structure, the agent learns to prioritize mission completion while maximizing data freshness.
Future research directions include transitioning from discrete grid-based movement to continuous action spaces, coordinating multiple UAVs for large-scale deployments, and optimizing flight speed and altitude to further enhance energy efficiency.
Summary updated