Abstract
Millimeter-wave (mmWave) communications are pivotal for high-bandwidth 5G and 6G networks, yet they rely heavily on precise beamforming to overcome propagation losses. In dynamic, mobile environments, maintaining optimal beam alignment is challenged by time-varying channels and the need to limit interference to neighboring users. This paper presents R-GP-UCB, a novel online learning algorithm that treats beam alignment as a non-stationary constrained bandit problem. By integrating Gaussian Process (GP) regression with a primal-dual optimization framework and a periodic restart mechanism, the proposed approach achieves sublinear regret and constraint violations, ensuring efficient and safe communication in highly mobile scenarios.
1. Introduction
The deployment of mmWave technology necessitates directional beamforming to concentrate signal energy. However, traditional beam alignment methods often struggle with three concurrent challenges: high sample complexity, non-stationary environments due to user mobility, and strict interference constraints in dense network deployments. Exhaustive beam sweeping is often too slow for real-time applications, and ignoring the time-varying nature of the channel leads to rapid performance degradation.
To address these issues, this research proposes a learning-based framework that does not require prior knowledge of the channel physics or user trajectories. Instead, it learns the optimal beam configuration on-the-fly while strictly adhering to interference thresholds.
2. Methodology: The R-GP-UCB Algorithm
The core of the proposed solution is the Restart GP-UCB with Constraints (R-GP-UCB) algorithm. It is built upon three foundational pillars designed to balance exploration, exploitation, and safety in dynamic settings.
2.1 Kernelized Bandits (Gaussian Processes)
Rather than treating each possible beam direction as an independent entity, R-GP-UCB utilizes Gaussian Processes within a Reproducing Kernel Hilbert Space (RKHS). This allows the algorithm to exploit the spatial correlation between beams; if a specific beam direction yields high signal strength, nearby directions are likely to perform similarly. This kernelized approach significantly reduces the sample complexity required to find the optimal beam.
2.2 Primal-Dual Optimization
To manage interference constraints, the algorithm employs a primal-dual framework. It maintains a dual variable that acts as a dynamic penalty weight. If a chosen beam violates interference constraints, the dual variable increases, effectively penalizing that region of the beam space in subsequent iterations. This ensures that the algorithm prioritizes "safe" beams that provide sufficient signal strength without exceeding interference limits.
2.3 Periodic Restarts
In non-stationary environments, historical data can become a liability if the channel state changes abruptly. R-GP-UCB incorporates a periodic restart strategy, where the algorithm's memory is wiped at specific intervals. This allows the learner to adapt to new channel conditions (e.g., a user moving behind an obstacle or a sudden change in path loss) without being biased by outdated observations.
3. Theoretical Contributions
A significant contribution of this work is the development of a novel per-time-slot bound for constraint violations. In stationary settings, researchers often rely on time-average metrics to prove safety. However, the authors demonstrate that these metrics are insufficient for time-varying channels. They prove that R-GP-UCB achieves a sublinear growth rate of $O(T^{3/4})$ for both cumulative regret (the difference between the algorithm’s performance and an omniscient optimum) and cumulative constraint violations. This theoretical guarantee ensures that the algorithm's average performance converges toward the optimum over time.
4. Performance Evaluation
The algorithm was evaluated using Matlab’s Phased Array System Toolbox, simulating a 60GHz carrier frequency with a 4-antenna uniform linear array. The channel model included one line-of-sight (LOS) path and ten non-line-of-sight (NLOS) paths to reflect realistic multipath environments.
4.1 Abruptly-Changing Environments
In scenarios where the channel characteristics jump at specific intervals, R-GP-UCB demonstrated superior resilience compared to standard constrained GP-UCB. While the baseline algorithm's regret spiked and remained high following a channel shift, R-GP-UCB successfully reset its learning state and quickly identified the new optimal beam direction.
4.2 Slowly-Varying Environments
In scenarios involving continuous user movement, the algorithm maintained consistently lower average regret and kept interference violations near zero. The results indicate that even without precise knowledge of the "variation budget" (the degree of channel change), the algorithm remains robust, though performance can be further optimized if the variation budget is known.
5. Conclusion and Future Work
R-GP-UCB provides a robust framework for beam alignment in the presence of mobility and interference. By treating the channel as a "black box," it offers a flexible solution that adapts to complex environments without requiring detailed physical models.
Future research directions include the development of adaptive restart intervals based on change-detection algorithms to improve efficiency over fixed intervals. Additionally, addressing the computational complexity of Gaussian Process regression—specifically the matrix inversions that scale with sample size—remains a priority for scaling the algorithm to massive MIMO systems with thousands of potential beam candidates.
Summary updated