Deep progressive reinforcement learning-based flexible resource scheduling framework for IRS and UAV-assisted MEC system

Li Dong, Feibo Jiang, Minjie Wang, Yubo Peng, Xiaolong Li

Introduction

Mobile Edge Computing (MEC) has emerged as a critical technology for enabling resource-constrained User Equipments (UEs) to offload computationally intensive tasks to nearby edge servers. To enhance the flexibility and coverage of MEC, researchers have integrated Unmanned Aerial Vehicles (UAVs) as mobile base stations and Intelligent Reflecting Surfaces (IRS) to mitigate signal blockage in dense urban environments. However, managing resources in such dynamic systems presents significant challenges, particularly when the number of serving UAVs fluctuates.

System Model and Problem Formulation

The system considers a multi-UAV and IRS-assisted MEC network where UEs offload tasks via wireless links. The presence of buildings often obstructs direct communication, necessitating the use of IRS to create virtual line-of-sight paths.

The IRS and UAV-assisted MEC system model, illustrating the interaction between UEs, UAVs, and reflecting surfaces.

The primary objective is to minimize total system energy consumption by optimizing discrete offloading decisions (user association) and continuous resource allocation (power and bandwidth). This results in a complex Mixed Integer Nonlinear Programming (MINLP) problem. Traditional Deep Reinforcement Learning (DRL) approaches often fail in dynamic scenarios where the number of UAVs changes, leading to "catastrophic forgetting" when the network architecture is modified.

The FRES Framework

To address these challenges, the authors propose the Flexible REsource Scheduling (FRES) framework. FRES leverages deep progressive reinforcement learning to maintain high performance across varying network configurations.

The FRES framework architecture, featuring a multi-task agent, progressive scheduler, and Light Taboo Search (LTS) refinement.

The framework consists of three core components:

  1. Multi-task Agent: A single neural network with a shared backbone that branches into a classification head for user association and a fitting head for resource allocation.
  2. Progressive Scheduler: This component allows the network to adapt to changes in the number of UAVs by adding or removing neurons without losing previously learned knowledge. This "progressive" expansion enables the system to switch between configurations in milliseconds.
  3. Light Taboo Search (LTS): A refinement stage that uses a lightweight local search algorithm to improve the actions generated by the DRL agent, ensuring the system avoids local minima and approaches a global optimum.

Performance Evaluation and Key Insights

The FRES framework was evaluated in a simulated urban environment with up to 50 UEs and 5 UAVs.

Energy Efficiency and Adaptation

FRES achieved a superior average energy consumption of 56.37 units, compared to 61.55 for DDPG and 60.09 for SAC. Notably, when the number of UAVs changed (e.g., from 3 to 4 and back to 3), FRES demonstrated remarkable stability. While traditional models required extensive retraining (over 50 seconds) to recover performance, FRES adapted in just 0.43 seconds by recalling parameters from its progressive structure.

Hybrid Intelligence

The results highlight the effectiveness of combining DRL with classical heuristics. While the DRL agent handles the high-dimensional state space to find a strong initial policy, the LTS refinement significantly lowers the final energy state, proving that hybrid approaches are well-suited for complex MINLP problems in wireless communications.

Conclusion

The FRES framework provides a robust solution for resource scheduling in dynamic IRS-UAV-assisted MEC systems. By employing a progressive neural architecture, it overcomes the limitation of catastrophic forgetting, allowing for seamless adaptation to varying network scales. Future research will focus on model compression and pruning to manage the potential parameter growth associated with progressive learning in even larger-scale deployments.

Paper Citation: Li Dong, et al. "Deep progressive reinforcement learning-based flexible resource scheduling framework for IRS and UAV-assisted MEC system." arXiv:2408.01248v1 [cs.LG] 2 Aug 2024.

Summary updated

BibTeX
@article{dong2025deep,
  author = {Li Dong and Feibo Jiang and Minjie Wang and Yubo Peng and Xiaolong Li},
  title = {{Deep progressive reinforcement learning-based flexible resource scheduling framework for IRS and UAV-assisted MEC system}},
  journal = {IEEE Trans. Neural Networks Learn. Syst.},
  year = {2025},
  doi = {10.1109/tnnls.2023.3341067},
}
Figure