Abstract
In this paper, a novel framework is proposed to optimize the downlink multi-user communication of a millimeter wave base station, which is assisted by a reconfigurable intelligent reflector (IR). In particular, a channel estimation approach is developed to measure the channel state information (CSI) in real-time. First, for a perfect CSI scenario, the precoding transmission of the BS and the reflection coefficient of the IR are jointly optimized, via an iterative approach, so as to maximize the sum of downlink rates towards multiple users. Next, in the imperfect CSI scenario, a distributional reinforcement learning (DRL) approach is proposed to learn the optimal IR reflection and maximize the expectation of downlink capacity. In order to model the transmission rate's probability distribution, a learning algorithm, based on quantile regression (QR), is developed, and the proposed QR-DRL method is proved to converge to a stable distribution of downlink transmission rate. Simulation results show that, in the error-free CSI scenario, the proposed approach yields over 30% and 2-fold increase in the downlink sum-rate, compared with a fixed IR reflection scheme and direct transmission scheme, respectively. Simulation results also show that by deploying more IR elements, the downlink sum-rate can be significantly improved. However, as the number of IR components increases, more time is required for channel estimation, and the slope of increase in the IR-aided transmission rate will become smaller. Furthermore, under limited knowledge of CSI, simulation results show that the proposed QR-DRL method, which learns a full distribution of the downlink rate, yields a better prediction accuracy and improves the downlink rate by 10% for online deployments, compared with a Q-learning baseline.
Introduction
Millimeter-wave (mmW) communications are a cornerstone of 5G and future wireless networks, providing the massive bandwidth required for high-speed data transfer. However, mmW signals suffer from high propagation loss and are easily blocked by physical obstacles like buildings or trees. When the direct line-of-sight (LOS) path is obstructed, signal quality drops precipitously. To address this, Reconfigurable Intelligent Reflectors (IRs)—metasurfaces capable of electronically steering reflected beams—have emerged as a promising solution to create virtual LOS paths and bypass blockages.
System Model
The proposed system consists of a multi-antenna base station (BS) serving multiple single-antenna users (UEs) with the assistance of an IR containing $N$ elements. The IR is controlled by an IR controller that communicates with the BS to coordinate reflection patterns.
As shown in Figure 1, the IR acts as a relay that reflects the mmW downlink signal from the BS to users who may be in a non-line-of-sight (NLOS) region relative to the BS. The primary challenge lies in optimizing the BS precoding and the IR reflection coefficients to maximize the sum-rate, especially when channel state information (CSI) is imperfect.
Optimization Framework
The study addresses two primary scenarios for channel knowledge:
1. Perfect CSI Scenario
When the channel is perfectly known, the authors employ a joint optimization approach. They iteratively refine the BS precoding and the IR reflection coefficients using Lagrangian dual transforms and fractional programming. This ensures that the beamforming at the BS and the phase shifts at the IR are perfectly aligned to maximize the downlink sum-rate.
2. Imperfect CSI Scenario: Distributional RL
In practical deployments, CSI is often noisy or outdated. Traditional Reinforcement Learning (RL) methods like Q-learning typically estimate the mean expected reward. However, wireless channels are inherently stochastic. The authors propose a Quantile Regression Distributional RL (QR-DRL) approach. Instead of a single point estimate, QR-DRL learns the entire probability distribution of the transmission rate. By modeling the "shape" of uncertainty, the IR controller can make more robust decisions that account for the variance and risk associated with different reflection configurations.
Channel Estimation and Timing Overhead
A critical aspect of IR-aided systems is the overhead required for channel estimation. The authors propose a protocol that divides the coherence time into specific phases.
As illustrated in Figure 2, the coherence time $\tau$ is partitioned into $N$ sub-phases for channel estimation ($\tau_c$), a processing and feedback phase ($\tau_m$), and finally the data transmission phase ($\tau_d$). This structure highlights a key trade-off: while increasing the number of IR elements ($N$) improves beamforming gain, it also increases the total time spent on estimation ($N \tau_c$), potentially reducing the time available for actual data transmission.
Key Results and Insights
Numerical simulations conducted at 30 GHz demonstrate significant performance gains:
- Sum-Rate Improvements: In perfect CSI conditions, the proposed optimization achieves a 30% gain over fixed reflectors and a 100% (2-fold) increase compared to direct transmission without an IR.
- Robustness of QR-DRL: Under imperfect CSI, the QR-DRL method outperforms standard Q-learning by 10%, proving that modeling the full distribution of rates leads to better online adaptation.
- Scaling Effects: While adding more IR elements generally improves the rate, the gains eventually plateau due to the increasing channel estimation overhead shown in the timing model.
Conclusion
This research provides a comprehensive framework for optimizing IR-assisted mmW networks. By combining iterative optimization for known channels and distributional RL for uncertain environments, the system achieves substantial capacity gains. Future work will explore extending this model to multi-antenna users (MIMO) and accounting for hardware constraints like discrete phase shifts.
Summary updated