Introduction
As the telecommunications industry moves toward 6G, the primary objective of wireless network optimization is shifting. While previous generations focused heavily on throughput and "sum-rate maximization," emerging applications like autonomous driving, tactile internet, and holographic communication demand ultra-low latency. Traditional Radio Resource Management (RRM) strategies often create a "rich get richer" scenario: users with strong channel conditions receive the most power to boost total network speed, while users with poor channels suffer from massive packet backups and high queueing delays.
This paper introduces a novel power allocation strategy for Device-to-Device (D2D) networks that prioritizes minimizing packet delay. By integrating Graph Neural Networks (GNNs) into a Reinforcement Learning (RL) framework, the authors address the dual challenges of interference complexity and model generalization.
The Problem: Beyond Throughput
In D2D networks, interference patterns are highly dynamic and difficult to model using traditional optimization. Furthermore, most machine learning models for RRM are brittle; a model trained for a network of six users typically fails when deployed in a network of twenty. To achieve 6G latency targets, a solution must be both delay-aware and scalable across varying network densities.
Methodology: Graph Reinforcement Learning
The authors propose a Graph Reinforcement Learning approach, modeling the D2D network as a directed graph where nodes represent D2D pairs and edges represent the interference between them.
GNN-Embedded PPO
Rather than using a GNN as a simple pre-processor, the architecture embeds GNN layers directly into both the Actor (policy) and the Critic (value function) of a Proximal Policy Optimization (PPO) algorithm. This integration allows for joint gradient backpropagation, leading to more efficient learning of the relationship between network topology and optimal power levels.
Delay-Awareness and Reward Design
Unlike standard models that rely solely on Channel State Information (CSI), this agent observes the "buffer state." This includes the number of backlogged packets and the delay of currently transmitted packets. The authors proved mathematically that a reward function based on the negative number of backlogged packets is asymptotically equivalent to minimizing the average delay across the entire network.
Permutation Equivariance and Scalability
Because GNNs process information based on local neighborhood structures rather than fixed indices, the model is naturally "permutation equivariant." This means the model learns the underlying physics of interference—how a neighbor's power affects a local rate—allowing it to scale to larger, unseen network topologies without retraining.
Experimental Setup
The framework was tested using high-fidelity simulations following 3GPP standards:
- Environment: A 500m × 500m area with pedestrian mobility (1 m/s).
- Channel Model: Dual-slope path-loss with 7 dB log-normal shadowing and Rayleigh fading.
- Traffic: Poisson arrival model (average 3 packets/ms).
- Baselines: The model was compared against WMMSE (optimization standard), ITLinQ, Max Power, and standard RL variants like TD3.
Key Results
- Significant Delay Reduction: The proposed method achieved a 56% improvement in average delay compared to the ITLinQ baseline.
- Improved Fairness: In 95th percentile delay tests (representing the worst-case user experience), the GRL approach dramatically outperformed WMMSE and Max Power, which saw delays spiral as traffic increased.
- Zero-Retraining Scalability: A model trained on only 6 D2D pairs performed effectively when tested on 54 pairs. This demonstrates a massive advantage for practical deployments where network density fluctuates.
- The Importance of Buffer Awareness: The study confirmed that CSI alone is insufficient for delay optimization. Knowing queue fullness allows the agent to make strategic sacrifices—briefly delaying a clear channel to help a congested one clear its backlog.
Conclusion and Future Directions
This research demonstrates that integrating GNNs with PPO creates a robust, scalable solution for delay-sensitive D2D networking. While the current model assumes a centralized controller with global knowledge, it sets the stage for future work in distributed GNN-based RL. Moving toward a system where nodes only exchange information with immediate neighbors could further reduce signaling overhead and increase resilience in the high-stakes environment of 6G communications.
Summary updated