A Non-stochastic Learning Approach to Energy Efficient Mobility Management

Cong Shen, Cem Tekin, Mihaela van der Schaar

Abstract

Energy efficient mobility management is an important problem in modern wireless networks with heterogeneous cell sizes and increased nodes densities. We show that optimization-based mobility protocols cannot achieve long-term optimal energy consumption, particularly for ultra-dense networks (UDN). To address the complex dynamics of UDN, we propose a non-stochastic online-learning approach which does not make any assumption on the statistical behavior of the small base station (SBS) activities. In addition, we introduce handover cost to the overall energy consumption, which forces the resulting solution to explicitly minimize frequent handovers. The proposed Batched Randomization with Exponential Weighting (BREW) algorithm relies on batching to explore in bulk, and hence reduces unnecessary handovers. We prove that the regret of BREW is sublinear in time, thus guaranteeing its convergence to the optimal SBS selection. We further study the robustness of the BREW algorithm to delayed or missing feedback. Moreover, we study the setting where SBSs can be dynamically turned on and off. We prove that sublinear regret is impossible with respect to arbitrary SBS on/off, and then develop a novel learning strategy, called ranking expert (RE), that simultaneously takes into account the handover cost and the availability of SBS. To address the high complexity of RE, we propose a contextual ranking expert (CRE) algorithm that only assigns experts in a given context. Rigorous regret bounds are proved for both RE and CRE with respect to the best expert. Simulations show that not only do the proposed mobility algorithms greatly reduce the system energy consumption, but they are also robust to various dynamics which are common in practical ultra-dense wireless networks.

Introduction

As wireless networks evolve toward 5G and beyond, the deployment of Ultra-Dense Networks (UDN) has become a primary strategy for meeting capacity demands. In these environments, User Equipment (UE) is often surrounded by a high density of Small Base Stations (SBSs), making traditional mobility management—deciding which cell should serve a user—increasingly complex.

Network model of a UDN where a user (UE 1) is surrounded by multiple candidate SBSs.

Traditional protocols, such as those defined by 3GPP, typically rely on myopic optimization, focusing on instantaneous signal strength. However, this approach often leads to the "Ping-Pong Effect," where frequent, unnecessary handovers (FHO) consume excessive energy. Furthermore, the unpredictable dynamics of UDNs—including fluctuating traffic loads and SBSs dynamically turning on or off—defy standard statistical modeling.

Proposed Methodology

This work shifts the paradigm from myopic optimization to long-term online learning using a non-stochastic framework. Unlike previous methods that assume specific statistical distributions for network behavior, the proposed approach treats the environment as potentially adversarial, ensuring robustness across diverse scenarios.

Comparison of the proposed non-stochastic learning approach with existing methods.

BREW: Batched Randomization with Exponential Weighting

The core contribution is the BREW algorithm, which adapts the Multi-Armed Bandit (MAB) framework for mobility management. BREW introduces two critical mechanisms:

  1. Batching: Time is divided into batches ($\tau$). The UE remains connected to a selected SBS for the entire batch duration. This "bulk exploration" naturally penalizes and reduces frequent handovers.
  2. Exponential Weighting: Based on the EXP3 algorithm, BREW maintains a probability distribution over SBSs, updating weights based on observed energy consumption.

Handling Dynamic SBS Availability

To address scenarios where SBSs may be unavailable (the "sleeping bandit" problem), the authors introduce Ranking Expert (RE) and Contextual Ranking Expert (CRE) algorithms. While they prove that sublinear regret is impossible for arbitrary SBS on/off patterns, these expert-based strategies provide a robust practical solution. CRE is particularly efficient as it uses the current serving cell as a "context" to limit the search space for the next optimal connection, significantly reducing computational overhead.

Key Insights

  • Handover Cost Integration: By explicitly including a switching penalty ($E_s$) in the utility function, the algorithm learns to prioritize stable, long-term connections over transient signal peaks.
  • Non-Stochastic Robustness: The assumption of a non-stochastic environment allows the system to remain performant even under worst-case interference or unpredictable user mobility.
  • Proven Convergence: The authors provide rigorous proofs that BREW achieves "sublinear regret" ($O(T^{2/3})$), meaning its performance asymptotically approaches that of an ideal "genie" with perfect foresight.

Performance Evaluation

The algorithms were tested using a system-level simulator following 3GPP TR 36.814 specifications. Key results include:

  • Energy Efficiency: BREW achieved 20%–30% lower energy consumption than standard 3GPP protocols in the short term, with gains exceeding 60% as the algorithm converged over time.
  • Resilience: The system demonstrated high robustness to delayed or missing feedback, which is critical for real-world hardware implementations.

Conclusion and Future Work

The proposed non-stochastic learning approach offers a significant leap in energy-efficient mobility management for UDNs. While BREW and CRE show immense promise, future work may focus on "anytime" versions of these algorithms that do not require a fixed time horizon $T$ for parameter tuning, as well as further optimizing computational complexity for low-power IoT devices.

Summary updated

Abstract

Energy efficient mobility management is an important problem in modern wireless networks with heterogeneous cell sizes and increased nodes densities. We show that optimization-based mobility protocols cannot achieve long-term optimal energy consumption, particularly for ultra-dense networks (UDN). To address the complex dynamics of UDN, we propose a non-stochastic online-learning approach which does not make any assumption on the statistical behavior of the small base station (SBS) activities. In addition, we introduce handover cost to the overall energy consumption, which forces the resulting solution to explicitly minimize frequent handovers. The proposed Batched Randomization with Exponential Weighting (BREW) algorithm relies on batching to explore in bulk, and hence reduces unnecessary handovers. We prove that the regret of BREW is sublinear in time, thus guaranteeing its convergence to the optimal SBS selection. We further study the robustness of the BREW algorithm to delayed or missing feedback. Moreover, we study the setting where SBSs can be dynamically turned on and off. We prove that sublinear regret is impossible with respect to arbitrary SBS on/off, and then develop a novel learning strategy, called ranking expert (RE), that simultaneously takes into account the handover cost and the availability of SBS. To address the high complexity of RE, we propose a contextual ranking expert (CRE) algorithm that only assigns experts in a given context. Rigorous regret bounds are proved for both RE and CRE with respect to the best expert. Simulations show that not only do the proposed mobility algorithms greatly reduce the system energy consumption, but they are also robust to various dynamics which are common in practical ultra-dense wireless networks.
BibTeX
@article{shen2016non,
  author = {Cong Shen and Cem Tekin and Mihaela van der Schaar},
  title = {{A Non-stochastic Learning Approach to Energy Efficient Mobility Management}},
  journal = {IEEE J. Sel. Areas Commun.},
  year = {2016},
  doi = {10.1109/jsac.2016.2612038},
}
Figure