Energy Saving in 6G O-RAN Using DQN-Based xApp

Qiao Wang, Swarna Bindu Chetty, Ahmed Al-Tahmeesschi, Xuanyu Liang, Yi Chu, Hamed AhmadiCamad 2024 · DOI

TL;DR

This paper tackles the critical challenge of energy saving in future 6G Open Radio Access Networks (O-RAN) by proposing a novel Deep Q-Network (DQN)-based approach implemented as an xApp. Their method dynamically switches off underutilized Radio Cards (RCs) to significantly reduce energy consumption while maintaining Quality of Service (QoS), outperforming existing heuristic and baseline methods.

What Problem Does This Paper Address?

The relentless growth of mobile networks, particularly with the advent of 5G and the anticipated 6G, has led to a massive increase in energy consumption. This is exacerbated by network densification, where more base stations (or Radio Units - RUs) are deployed. Traditional energy-saving methods struggle to keep pace with the dynamic nature of these networks. The Open RAN (O-RAN) architecture, with its emphasis on openness and intelligence, offers a fertile ground for AI/ML-driven solutions. However, efficiently managing the ON/OFF states of Radio Cards (RCs) within O-RAN to save energy, without compromising user experience (QoS), is a complex optimization problem. This paper aims to address this by developing an intelligent, automated solution using Deep Reinforcement Learning (DRL).

Methods

The core of the proposed solution is a Deep Q-Network (DQN) algorithm, a type of Deep Reinforcement Learning. The authors frame the problem as a Markov Decision Process (MDP) where the agent (the xApp) learns to make optimal decisions about switching RCs ON or OFF.

  • DQN-Based Approach: Instead of traditional Q-learning which struggles with large state spaces, DQN uses a neural network to approximate the Q-values, making it suitable for complex environments like O-RAN.
  • xApp Implementation: The DQN model is implemented as an "xApp" (application) within the O-RAN Intelligent Controller (RIC). This allows it to integrate seamlessly into the O-RAN ecosystem.
  • Two ES-xApp Variants:
    • ES-xApp-1: This variant uses a richer state representation, incorporating both Received Signal Strength (RSS) and User Equipment (UE) geolocations.
  • ES-xApp-2: This variant uses a simpler state representation, relying solely on RSS information.
  • State Space: The state observed by the DQN includes information about the network, specifically RSS matrices and, for ES-xApp-1, UE coordinates.
  • Action Space: The action space is binary for each RC: either ON or OFF.
  • Reward Function: The reward function is carefully designed to incentivize energy saving (maximizing switched-off RCs) while penalizing violations of QoS constraints (RSS below a minimum threshold or RC capacity exceeded).

Datasets

The paper does not explicitly mention using pre-existing public datasets for training or evaluation. Instead, they developed a custom O-RAN simulator in Python 3.10 to generate the necessary network data and evaluate their proposed ES-xApps. The simulation parameters are detailed in Table I, including the number of UEs, RUs, RCs, channel model, transmission power, frequency bands, and simulation area.

Key Insights and Ideas

  • The Power of Richer State Information: ES-xApp-1, which incorporates UE geolocations in addition to RSS, demonstrated more stable training and faster convergence compared to ES-xApp-2 (RSS only). This suggests that providing more contextual information to the RL agent significantly aids its learning process.
  • DRL as a Viable Solution for Complex Optimization: The paper successfully demonstrates that DRL can effectively tackle the NP-hard problem of dynamic RC ON-OFF scheduling, outperforming traditional heuristic approaches.
  • O-RAN as an Enabler for Intelligent Energy Saving: The O-RAN architecture, with its open interfaces and RIC, is crucial for enabling such intelligent, AI-driven energy-saving mechanisms by facilitating data collection and action execution.
  • Trade-off Management: The carefully designed reward function highlights the importance of balancing energy savings with QoS, a critical aspect for real-world network deployment.

Key Results

The proposed DQN-based ES-xApps significantly outperformed both a heuristic and a baseline xApp in terms of energy saving.

  • RC Switch-off Performance: With 50 UEs, the ES-xApp-1 managed to switch off 50% of the Radio Cards (RCs), a substantial improvement over the heuristic algorithm's 17%.
  • Stability and Robustness: ES-xApp-1 showed a much smaller standard deviation in its performance across simulations, indicating greater stability and less variability compared to other methods, especially with a higher number of UEs.
  • Training Convergence: ES-xApp-1 exhibited faster and more stable convergence during training, attributed to the richer input state.

Limitations and Future Work

  • Simulation-Based Evaluation: The evaluation was conducted in a simulated O-RAN environment. While the simulator is described, real-world deployment and testing are necessary to validate these findings in a live network.
  • Specific Network Conditions: The study focused on a small-cell network with specific parameters. Future work could explore performance under different network topologies, traffic patterns, and mobility scenarios.
  • Scalability: While the DQN approach is designed to handle large state spaces, further investigation into its scalability with an even greater number of RUs and UEs would be beneficial.
  • Dynamic UE Behavior: The current model assumes static UEs. Incorporating dynamic UE mobility and handover scenarios would add further realism.
  • Other Energy Saving Strategies: The paper focuses on RC ON-OFF. Future research could explore integrating this with other energy-saving techniques within the O-RAN framework.

Summary updated

BibTeX
@inproceedings{wang2024energy,
  author = {Qiao Wang and Swarna Bindu Chetty and Ahmed Al-Tahmeesschi and Xuanyu Liang and Yi Chu and Hamed Ahmadi},
  title = {{Energy Saving in 6G O-RAN Using DQN-Based xApp}},
  booktitle = {Camad},
  year = {2024},
  doi = {10.1109/camad62243.2024.10942980},
}
Figure