DLB: Deep Learning Based Load Balancing

Xiao-ke Zhu, Qi Zhang, Taining Cheng, Ling Liu, Wei Zhou, Jing HeCLOUD 2019 · OpenReview

Introduction

In modern cloud computing, efficient load balancing is critical for preventing server bottlenecks and ensuring high throughput. Most contemporary systems utilize Consistent Hashing to distribute workloads. While mathematically elegant and efficient for uniform data, these static hash functions are "distribution-blind." When faced with real-world "skewed" data—such as web traffic or sensor logs following power-law distributions—traditional hashes produce unbalanced outputs. This creates "hotspots" where specific servers are overwhelmed while others remain underutilized, leading to increased latency and wasted resources.

The DLB Framework

To address these limitations, we propose DLB (Deep Learning Based Load Balancing). DLB re-imagines load balancing as a learned mapping problem. Instead of a fixed mathematical formula, it employs a hierarchical architecture of neural networks to transform skewed input distributions into a uniform output space.

The hierarchical architecture of the DLB framework, illustrating the flow from input to the final hash circle.

The DLB pipeline consists of four primary stages:

  1. Input Stage: Raw data (e.g., IP addresses, object IDs) is vectorized for neural network processing.
  2. Disperse Stage: To manage complexity, DLB uses a tree-like hierarchy. A root model directs inputs to specific sub-models, effectively partitioning the distribution space.
  3. Mapping Stage: Leaf models generate "local" positions within specific segments of the hash space.
  4. Join Stage: Local coordinates are translated into a global position on a $2^{32}-1$ hash circle.

By using a hierarchy, DLB avoids the computational burden of training a single massive model, allowing for faster convergence and more precise mapping of complex data patterns.

Experimental Evaluation

We evaluated DLB against standard Consistent Hashing (CH) and Consistent Hashing with Bounded Loads (CHBL) using both synthetic (Uniform, Normal, Log-normal) and real-world datasets (4TB radio monitoring data). Simulations were conducted in CloudSim with a 64-machine environment.

Key Results

DLB consistently outperformed traditional methods in both load distribution and execution speed. On real-world data, the standard deviation of server loads for traditional hashes was nearly 3x higher than that of DLB.

Performance of DLB in a simulated environment, showing a significantly reduced maximum job duration of 880.1 seconds compared to traditional methods.

The impact on job completion time was particularly striking:

  • Log-normal distribution: DLB achieved a 61.3% reduction in maximum job duration compared to CH.
  • Normal distribution: DLB reduced execution time by 33%.
  • Uniform distribution: Even without skew, DLB maintained a ~60% performance lead, demonstrating that the overhead of model inference does not negate its balancing benefits.

Insights and Future Work

The success of DLB stems from its ability to leverage the temporal consistency of data patterns. While data is often skewed, the nature of that skew tends to be stable for specific applications, making historical data an excellent training signal.

However, several challenges remain. The current implementation requires offline training, and future work should explore online learning to adapt to shifting distributions in real-time. Additionally, the complexity of managing a neural network hierarchy may be high for smaller deployments. Despite these hurdles, DLB represents a significant step toward "intelligent" infrastructure that learns from its environment to optimize performance.

Summary updated

Abstract

In this paper, we introduce DLB, a Deep Learning based load Balancing mechanism, to effectively address the data skew problem. The key idea of DLB is to replace hash functions in the load balancing mechanisms with deep learning models, which are trained to be able to map different distributions of workloads and data to the servers in a uniformed manner. We implemented DLB and deployed it on a practical Cloud environment using CloudSim. Experimental results using both synthetic and real-world data sets show that compared with traditional hash function based load balancing methods, DLB is able to achieve more balanced mappings, especially when the workload is highly skewed.
BibTeX
@inproceedings{zhu2019dlb,
  author = {Xiao-ke Zhu and Qi Zhang and Taining Cheng and Ling Liu and Wei Zhou and Jing He},
  title = {{DLB: Deep Learning Based Load Balancing}},
  booktitle = {CLOUD},
  year = {2019},
  url = {https://openreview.net/forum?id=DdYWm-87zt},
}
Figure