Multi-Heterogeneous-Agent DRL for Efficient Congestion Control With Reward Redistribution in Space-Air-Ground Integrated Networks

Jinsong Gui, Zhengyang Li, Jingjing Zhang, Xiaoheng Deng, Geyong Min

Abstract

As a core architecture in sixth-generation (6 G) communication systems, space–air–ground integrated networks (SAGINs) provide seamless global coverage. However, due to frequent link switching, rapid fluctuations in network resources, and long round-trip times (RTTs), state feedback is delayed, which substantially impairs the responsiveness and stability of conventional congestion control (CC) schemes. In parallel, large-scale heterogeneous active flows impose dynamic and often conflicting Quality-of-Service (QoS) demands, intensifying resource contention in constrained environments. These challenges are further compounded in scenarios with rapidly evolving cross-layer conditions and concurrent flow scheduling, where maintaining global state consistency and satisfying per-flow QoS requirements becomes particularly difficult. To address these issues, we propose MHA-DRL-CC, a CC scheme based on heterogeneous multi-agent deep reinforcement learning (DRL), equipped with proactive state prediction and reward redistribution mechanisms for SAGINs. The framework predicts the joint state of SAGIN, network, and transport layers, and extracts flow-level features to mitigate observation delays. It employs heterogeneous agents to make adaptive CC decisions for different flow types and redistributes rewards at a fine-grained level, thereby meeting personalized QoS requirements while ensuring fast convergence and robust performance. Extensive simulations across four representative scenarios show that MHA-DRL-CC consistently outperforms baseline schemes such as BBR, MACO, DRL-CC, and DRL-3R in highly dynamic link conditions, demonstrating superior efficiency and robustness in complex SAGIN environments.
BibTeX
@article{gui2026multi,
  author = {Jinsong Gui and Zhengyang Li and Jingjing Zhang and Xiaoheng Deng and Geyong Min},
  title = {{Multi-Heterogeneous-Agent DRL for Efficient Congestion Control With Reward Redistribution in Space-Air-Ground Integrated Networks}},
  journal = {IEEE Trans. Network Science \& Engineering},
  year = {2026},
  doi = {10.1109/tnse.2025.3625582},
}

Loading related content...

Figure