Adaptive Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks

Xu Zhu, Junyu He, Ming ZhaoInf 2025 · DOI

TL;DR

This paper tackles the challenge of optimizing a drone's flight path and actions in a wireless-powered IoT network to simultaneously maximize data collection, energy harvesting, and minimize its own energy consumption. They introduce a novel deep reinforcement learning approach, DW-MODDPG, that dynamically adjusts the drone's priorities based on real-time conditions, leading to more efficient and adaptable operations compared to fixed strategies.

What Problem Does This Paper Address?

The Internet of Things (IoT) is rapidly expanding, connecting billions of devices that generate vast amounts of data. However, many of these devices are energy-constrained and located in hard-to-reach places, making battery replacement impractical. Wireless Power Transfer (WPT) offers a solution, but efficiently combining data collection and energy delivery with a mobile platform like a drone presents a complex multi-objective optimization problem. Drones need to balance several conflicting goals: flying fast enough to collect data from many devices (maximizing data rate), hovering long enough to transfer energy (maximizing harvested energy), and flying efficiently to conserve their own battery (minimizing propulsion energy). Traditional methods struggle with the dynamic nature of these networks, where priorities can shift due to changing data demands or energy levels.

Methods

The authors propose a novel approach called Dynamic-Weight Multi-Objective Deep Deterministic Policy Gradient (DW-MODDPG). This method builds upon the Deep Deterministic Policy Gradient (DDPG) algorithm, a popular deep reinforcement learning technique for continuous control problems. The key innovation is the introduction of a dynamic-weight mechanism implemented by a neural network called WeightNet.

Here's how it works:

  • Vector-Valued Rewards: Instead of a single scalar reward, the drone receives a vector of rewards, each corresponding to a specific objective (data collection, energy harvesting, energy consumption, and an auxiliary reward for spatial guidance).
  • Adaptive Weighting: WeightNet takes the drone's current state (e.g., its position, operational boundaries, data backlog) as input and dynamically generates a set of weights. These weights determine how much importance the drone places on each objective at that specific moment.
  • Online Scalarization: The dynamically generated weights are used to combine the vector rewards into a single scalar reward, which is then used to train the DDPG agent. This allows the drone to adapt its strategy on the fly – for instance, prioritizing energy harvesting when its battery is low or focusing on data collection when there's a backlog.
  • Actor-Critic Architecture: The DW-MODDPG algorithm uses an actor-critic framework. The actor network learns the optimal policy (how to act), while the critic network learns to estimate the value of those actions, guiding the actor's learning process.

Datasets

Not discussed in the paper. The authors developed a simulation environment to evaluate their proposed algorithm.

Key Insights and Ideas

  • Dynamic Priorities are Crucial: The paper strongly emphasizes that fixed priorities for optimization objectives are insufficient in real-world UAV-assisted IoT scenarios. The ability to adapt these priorities based on the current state of the environment (e.g., device battery levels, data buffer status) is key to achieving optimal performance.
  • WeightNet as a "Contextual Prioritizer": The WeightNet acts as an intelligent "prioritizer," allowing the drone to make context-aware decisions. This is a significant departure from traditional methods that use static weights or complex Pareto front analysis.
  • Balancing Act is Achievable: The results demonstrate that it's possible to achieve a favorable balance between seemingly conflicting objectives like maximizing data rate and harvested energy while minimizing energy consumption, thanks to the adaptive weighting.
  • Generalizability: The DW-MODDPG framework is designed to be flexible and can handle an arbitrary number of objectives, suggesting its potential applicability to a wider range of complex optimization problems.

Key Results

The DW-MODDPG algorithm demonstrated superior performance compared to benchmark policies (fixed-speed heuristics).

  • Improved Performance: The proposed algorithm achieved up to a 13.8% increase in total harvested energy and up to a 5.4% increase in the sum data rate compared to a fixed-weight baseline.
  • Comparable or Lower Energy Consumption: Crucially, these improvements were achieved while maintaining comparable or even lower UAV energy consumption.
  • Adaptive Behavior: Simulations showed that the DW-MODDPG policy adaptively adjusted its hovering locations and flight strategies to balance competing objectives, leading to better overall outcomes across various scenarios. For instance, it could prioritize data collection when channels were strong or shift focus to energy harvesting when more devices were available within its coverage.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Perfect CSI Assumption: The current model assumes perfect Channel State Information (CSI). In reality, CSI estimation errors can degrade performance. Future work could explore robust or distributionally robust optimization techniques to handle these uncertainties.
  • Idealized Energy Harvesting: The model assumes ideal energy harvesting circuits. Real-world hardware has nonlinearities that can affect efficiency. Incorporating these nonlinearities and exploring their impact is a potential avenue for future research.
  • Single-UAV Scenario: The current study focuses on a single UAV. Extending the framework to multi-UAV systems introduces significant challenges like inter-UAV coordination, interference management, and the need for scalable learning algorithms. Future work could explore multi-agent deep reinforcement learning architectures.
  • Decentralized Learning: For enhanced scalability and privacy, exploring decentralized learning paradigms like federated reinforcement learning could be a valuable extension.

Summary updated

BibTeX
@article{zhu2025adaptive,
  author = {Xu Zhu and Junyu He and Ming Zhao},
  title = {{Adaptive Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks}},
  journal = {Inf.},
  year = {2025},
  doi = {10.3390/info16100849},
}
Figure