Constrained Deep Reinforcement Learning for Fronthaul Compression Optimization

Axel Grönland, Alessio Russo, Yassir Jedra, Bleron Klaiqi, Xavier Gelabert

Abstract

Centralized-Radio Access Networks (C-RAN) offer significant operational advantages for 5G/6G deployments but introduce a critical data bottleneck in the fronthaul (FH) segment. This paper presents a model-free, constrained Deep Reinforcement Learning (DRL) framework designed to dynamically optimize fronthaul compression. By formulating the problem as a Constrained Markov Decision Process (CMDP) and employing Lagrangian dual optimization, the proposed system balances throughput maximization against strict latency and packet loss requirements. Experimental results from a high-fidelity simulator demonstrate a 70% increase in fronthaul utilization compared to static reference schemes while strictly adhering to safety constraints.

Introduction

The evolution toward 5G and 6G architectures has popularized the Centralized-Radio Access Network (C-RAN) model. In this setup, baseband processing is shifted from individual cell sites to a centralized "cloud" pool of Baseband Units (BBUs). While this centralization facilitates better coordination and cost efficiency, it places immense pressure on the fronthaul (FH) link—the high-speed connection between the central pool and the Remote Radio Units (RRUs).

With the advent of Massive MIMO and wider bandwidths, the raw data rates on the FH link can exceed the capacity of standard fiber connections. Engineers employ various compression techniques, such as reducing modulation orders or quantizing precoding weights, to manage this load. However, excessive compression degrades signal quality, while insufficient compression leads to congestion, high latency, and packet loss. Finding the optimal compression level in a dynamic, time-varying network environment is a complex optimization challenge that traditional model-based approaches struggle to solve without perfect environmental knowledge.

System Architecture and Problem Formulation

The proposed architecture integrates an AI Compute Unit (ACU) within the BBU pool to manage the functional split between the Baseband High (BBH) and Baseband Low (BBL) layers.

Physical and logical architecture of the C-RAN system with AI-driven fronthaul optimization.

As shown in Figure 1, the ACU monitors the network state and adjusts "compression knobs" at the functional split. These knobs include the modulation order, precoder bitwidth (quantization), and sub-band precoding granularity. The goal is to maximize the average FH utility—a measure of effective throughput—while ensuring that the average FH latency remains below a predefined threshold (e.g., 260 $\mu$s) and packet loss is minimized.

The problem is framed as a Constrained Markov Decision Process (CMDP). Unlike standard RL, which only seeks to maximize a reward, CMDP agents must satisfy a set of cost constraints. This is particularly crucial in telecommunications, where violating latency requirements can lead to dropped calls or service failures.

Methodology: Constrained DRL

To solve the CMDP, the authors utilize Lagrangian Dual Optimization. The agent maximizes a modified reward function that is penalized by the constraints, while a "dual" variable (the Lagrange multiplier) is updated to ensure the constraints are strictly met.

A key innovation in this work is Value Function Decomposition. The agent learns separate value functions (Q-functions) for the reward and each individual constraint. This decomposition provides "model transparency," allowing operators to see exactly which factor—latency or packet loss—is driving the agent's current policy.

The framework was implemented using two DRL algorithms: Deep Q-Networks (DQN) for scenarios with a manageable action space and Soft Actor-Critic (SAC) with an auto-regressive policy to handle the exponential action space inherent in coordinating multiple cells simultaneously.

Experimental Evaluation and Results

The system was evaluated using the BasebAnd System Simulator (BASS), built on the ns-3 network simulator. The testbed simulated a 3-cell C-RAN environment with a 25 Gbps fronthaul capacity. Network traffic was modeled as a stochastic process, varying the number of Physical Resource Blocks (PRBs) to mimic realistic, fluctuating loads.

Average fronthaul utility and latency performance across different network loads (Mean RB).

As illustrated in Figure 3, both DQN and SAC agents significantly outperformed a conservative "Reference" scheme. The Reference scheme, designed for worst-case scenarios, maintains low latency but suffers from poor utilization as the network load increases. In contrast, the DRL agents dynamically adjusted compression levels to maintain high utilization (up to a 70.3% improvement) while keeping the average latency within the 260 $\mu$s safety limit. The agents also exhibited stable convergence, proving that they learned a reliable policy that respects network limits without sacrificing performance.

Conclusion and Future Work

This research highlights the potential of constrained DRL for managing complex trade-offs in 5G/6G fronthaul networks. By incorporating safety constraints directly into the learning process and providing explainable value decomposition, the proposed framework offers a "trustworthy AI" solution for network optimization. Future work will focus on scaling the implementation to handle dozens of cells and validating the framework on real-world hardware where processing jitter and non-ideal link behavior are more prevalent.

Summary updated

Abstract

In the Centralized-Radio Access Network (C-RAN) architecture, functions can be placed in the central or distributed locations. This architecture can offer higher capacity and cost savings but also puts strict requirements on the fronthaul (FH), these constraints can be any number of constraints but in this work we consider a constraint on packet loss and latency. Adaptive FH compression schemes that adapt the compression amount to varying FH traffic are promising approaches to deal with stringent FH requirements. In this work, we design such a compression scheme using a model-free off policy deep reinforcement learning algorithm which accounts for FH latency and packet loss constraints. Furthermore, this algorithm is designed for model transparency and interpretability which is crucial for AI trustworthiness in performance critical domains. We show that our algorithm can successfully choose an appropriate compression scheme while satisfying the constraints and exhibits a roughly 70% increase in FH utilization compared to a reference scheme.
BibTeX
@misc{gronland2023constrained,
  author = {Axel Grönland and Alessio Russo and Yassir Jedra and Bleron Klaiqi and Xavier Gelabert},
  title = {{Constrained Deep Reinforcement Learning for Fronthaul Compression Optimization}},
  year = {2023},
  eprint = {2309.15060},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.15060},
}
Figure