Abstract
In the Beyond 5G (B5G) era, network slicing has emerged as a critical technique for flexibly supporting diverse wireless services. By partitioning a single physical network into multiple independent slices, operators can cater to the distinct requirements of enhanced Mobile Broadband (eMBB), Ultra-Reliable Low-Latency Communications (URLLC), and massive Machine-Type Communications (mMTC). However, the combinatorial complexity of resource block (RB) allocation makes optimal slicing an NP-hard problem. This paper proposes a Deep Reinforcement Learning (DRL)-based network slicing technique (DRL-NS) that maximizes long-term throughput while strictly adhering to Quality of Service (QoS) constraints. The core innovation is an action elimination strategy that prunes the action space of undesirable decisions, significantly improving training efficiency and system performance.
The Challenge of Beyond 5G Network Slicing
B5G systems must simultaneously manage three service categories with conflicting goals:
- eMBB: Demands high data rates for applications like 4K streaming and AR/VR.
- URLLC: Requires near-instantaneous response times for autonomous vehicles and industrial automation.
- mMTC: Supports a massive density of low-power IoT sensors.
Traditional analytical methods struggle with the dynamic, time-varying nature of wireless channels. While standard DRL offers a potential solution, it often suffers from an astronomical action space. In a typical setup, a DRL agent might waste most of its training time exploring "illegal" resource allocations that violate latency or rate requirements, leading to slow convergence and suboptimal policies.
Proposed DRL-NS Framework
The proposed DRL-NS framework utilizes a Deep Q-Network (DQN) where the base station (gNB) serves as the intelligent agent. To overcome the exploration inefficiency of standard DRL, the authors introduce Action Space Reduction via Action Elimination.
Before the agent selects an action, the system applies two sequential filters:
- URLLC Test: Eliminates any RB allocation that cannot meet the strict maximum packet delay requirements for URLLC users.
- eMBB Test: Removes allocations that fail to satisfy the minimum data rate thresholds for eMBB users.
By narrowing the search to a "desirable action space," the agent only learns from feasible actions. The state space incorporates current and previous channel matrices to capture temporal correlations, while the reward function is tied to total system throughput, incentivizing efficiency within the "safe" filtered space.
Experimental Setup
The framework was evaluated in a high-fidelity simulation environment:
- Channel Modeling: Rayleigh fading for small-scale effects and the Hata-COST231 model for large-scale path loss.
- Network Topology: 10 small cells (gNBs) serving 4 User Equipments (UEs) with a 200m inter-site distance.
- Implementation: Developed using TensorFlow and trained on an NVIDIA GeForce Titan Xp.
Results and Performance Analysis
Numerical results demonstrate that DRL-NS significantly outperforms traditional heuristics and vanilla DRL.
Key findings include:
- Throughput Gains: DRL-NS achieved approximately 25% higher throughput than equal allocation and 15% higher than regression-tree-based methods.
- Training Efficiency: As shown in the cumulative reward plots, the action elimination strategy allows the agent to converge much faster than a standard DQN. DRL-NS outperformed vanilla DQN by 10% in final throughput, proving that the filtered search space leads to a superior policy.
- Robustness: The system maintained high performance even under stringent QoS requirements where heuristic methods typically fail.
Conclusion and Future Work
DRL-NS provides a robust solution for the complex resource allocation problems inherent in B5G network slicing. By integrating domain-specific constraints directly into the RL exploration process, the framework achieves high sample efficiency and superior throughput.
Future research directions include:
- Reducing Computational Overhead: Parallelizing the action filtering process to scale to larger numbers of RBs.
- Multi-Agent DRL (MADRL): Transitioning to a decentralized model where multiple gNBs coordinate autonomously.
- Uplink Integration: Extending the framework to handle uplink constraints, such as user device power limitations.
Summary updated