Introduction
In modern cloud computing, efficient load balancing is critical for preventing server bottlenecks and ensuring high throughput. Most contemporary systems utilize Consistent Hashing to distribute workloads. While mathematically elegant and efficient for uniform data, these static hash functions are "distribution-blind." When faced with real-world "skewed" data—such as web traffic or sensor logs following power-law distributions—traditional hashes produce unbalanced outputs. This creates "hotspots" where specific servers are overwhelmed while others remain underutilized, leading to increased latency and wasted resources.
The DLB Framework
To address these limitations, we propose DLB (Deep Learning Based Load Balancing). DLB re-imagines load balancing as a learned mapping problem. Instead of a fixed mathematical formula, it employs a hierarchical architecture of neural networks to transform skewed input distributions into a uniform output space.
The DLB pipeline consists of four primary stages:
- Input Stage: Raw data (e.g., IP addresses, object IDs) is vectorized for neural network processing.
- Disperse Stage: To manage complexity, DLB uses a tree-like hierarchy. A root model directs inputs to specific sub-models, effectively partitioning the distribution space.
- Mapping Stage: Leaf models generate "local" positions within specific segments of the hash space.
- Join Stage: Local coordinates are translated into a global position on a $2^{32}-1$ hash circle.
By using a hierarchy, DLB avoids the computational burden of training a single massive model, allowing for faster convergence and more precise mapping of complex data patterns.
Experimental Evaluation
We evaluated DLB against standard Consistent Hashing (CH) and Consistent Hashing with Bounded Loads (CHBL) using both synthetic (Uniform, Normal, Log-normal) and real-world datasets (4TB radio monitoring data). Simulations were conducted in CloudSim with a 64-machine environment.
Key Results
DLB consistently outperformed traditional methods in both load distribution and execution speed. On real-world data, the standard deviation of server loads for traditional hashes was nearly 3x higher than that of DLB.
The impact on job completion time was particularly striking:
- Log-normal distribution: DLB achieved a 61.3% reduction in maximum job duration compared to CH.
- Normal distribution: DLB reduced execution time by 33%.
- Uniform distribution: Even without skew, DLB maintained a ~60% performance lead, demonstrating that the overhead of model inference does not negate its balancing benefits.
Insights and Future Work
The success of DLB stems from its ability to leverage the temporal consistency of data patterns. While data is often skewed, the nature of that skew tends to be stable for specific applications, making historical data an excellent training signal.
However, several challenges remain. The current implementation requires offline training, and future work should explore online learning to adapt to shifting distributions in real-time. Additionally, the complexity of managing a neural network hierarchy may be high for smaller deployments. Despite these hurdles, DLB represents a significant step toward "intelligent" infrastructure that learns from its environment to optimize performance.
Summary updated