Intra- and Inter-Server Smart Task Scheduling for Profit and Energy Optimization of HPC Data Centers

Sayed Ashraf Mamun, Alexander Gilday, Amit Kumar Singh, Amlan Ganguly, Geoff V. Merrett, Xiaohang Wang, Bashir M. Al‐HashimiJournal of Low Power Electronics and Applications 2020 · DOI

TL;DR

This paper tackles the dual challenge of optimizing profit and energy consumption in High-Performance Computing (HPC) data centers. They introduce a novel reinforcement learning-based scheduler that dynamically allocates jobs to resources without requiring prior job profiling, achieving significant improvements in both profit and energy efficiency. Additionally, a network-aware server consolidation technique (BCC) is proposed to combat server underutilization, further reducing power consumption.

What Problem Does This Paper Address?

HPC data centers are notorious for their high power consumption, largely due to server over-provisioning and underutilization. Existing solutions often rely on hardware-based approaches like Dynamic Voltage and Frequency Scaling (DVFS) or Virtual Machine (VM) consolidation. However, these methods can be system-specific, require extensive simulations, or necessitate pre-runtime job profiling. The authors aim to bridge this gap by developing a flexible, intelligent scheduling system that can optimize for both economic gain (profit) and energy efficiency without needing advance knowledge of job characteristics. They also address the persistent problem of server underutilization, which contributes to wasted energy.

Methods

The core of the paper introduces two main innovations:

  • Intra-Server Smart Task Scheduling (CBA): This approach adapts the Multi-Armed Bandit (MAB) framework, specifically using an Upper Confidence Bound (UCB) inspired algorithm. Instead of relying on pre-defined job profiles, the system learns job value and energy consumption in real-time through exploration and exploitation. It models job allocation as a series of "actions" and learns to select the best action (job allocation) that maximizes the ratio of value to energy. This "Confidence-Based Approach" (CBA) is designed to handle dynamic job arrivals and varying resource availability.
  • Inter-Server Network-Aware Server Consolidation (BCC): To address server underutilization, the authors propose a Bandwidth-Constrained Consolidation (BCC) algorithm. This technique goes beyond simple consolidation by considering network traffic patterns. It clusters servers with high inter-communication and then intelligently migrates VMs between servers, prioritizing those that minimize network congestion and adhere to bandwidth constraints. A mathematical model is also derived to optimize the frequency of these consolidation operations, balancing power savings against network traffic overhead.

Datasets

The paper utilizes the Princeton Application Repository for Shared-Memory Computers (PARSEC) benchmark suite as the set of applications for job scheduling. Value curves for these applications were designed based on testing execution times across different numbers of cores. For the BCC algorithm's network traffic modeling, real data center traffic measurements from existing literature were used, with traffic patterns synthesized using Poisson, Pareto, and Gaussian distributions.

Key Insights and Ideas

  • Learning without Foresight: The most significant insight is the successful application of reinforcement learning (specifically, a UCB-inspired approach) to dynamically optimize job scheduling for both profit and energy without requiring pre-existing job profiles. This makes the system adaptable to diverse and unpredictable workloads.
  • Profit and Energy Synergy: The research demonstrates that optimizing for profit and energy simultaneously is not only possible but can lead to synergistic benefits, as seen in the CBA algorithm's performance.
  • Network Awareness is Crucial for Consolidation: Simply consolidating servers isn't enough. The BCC algorithm highlights that considering network bandwidth constraints is vital to prevent performance degradation and achieve true power savings, especially in high-traffic scenarios.
  • Optimizing Consolidation Frequency: The derivation of a mathematical model for optimal inter-consolidation time is a key contribution, allowing for more efficient and less disruptive consolidation cycles.

Key Results

  • CBA Scheduler: The proposed CBA scheduler increased profit by 40% and reduced energy consumption by 20% compared to a heuristic-based approach (HBA). It also resulted in 5% fewer zero-value jobs, indicating improved user satisfaction.
  • BCC Consolidation: The BCC consolidation technique reduced data center power consumption by up to 37%.
  • Combined Approach: When both CBA and BCC were combined, they achieved the maximum power reduction, particularly in wireless data center architectures.
  • Inter-Consolidation Time Model: The mathematical model for optimal inter-consolidation time was shown to be significantly faster (over 1000x speedup) than Monte Carlo simulations while providing accurate estimations.

Limitations and Future Work

  • Energy Measurement Granularity: The paper acknowledges that the energy measurement on the Odroid boards was for the entire processor, not individual cores. This means energy consumption for single jobs was an estimate and likely overestimated, potentially underestimating the reward.
  • VM Migration Granularity: The BCC algorithm consolidates at the server level rather than VM level, which is a simplification for scalability but might miss finer-grained optimization opportunities.
  • Parameter Optimization: While a confidence value (δ) was explored for the CBA algorithm, further in-depth optimization is suggested for systems with different reward distribution assumptions.
  • Future Work: The authors suggest further experimentation with different arrival rates and the exploration of more advanced reinforcement learning techniques. They also highlight the need for more accurate energy measurement at the core level for finer-grained analysis. The potential for combining CBA and BCC for even greater power savings, especially in wireless architectures, is also a promising avenue.

Summary updated

Abstract

Servers in a data center are underutilized due to over-provisioning, which contributes heavily toward the high-power consumption of the data centers. Recent research in optimizing the energy consumption of High Performance Computing (HPC) data centers mostly focuses on consolidation of Virtual Machines (VMs) and using dynamic voltage and frequency scaling (DVFS). These approaches are inherently hardware-based, are frequently unique to individual systems, and often use simulation due to lack of access to HPC data centers. Other approaches require profiling information on the jobs in the HPC system to be available before run-time. In this paper, we propose a reinforcement learning based approach, which jointly optimizes profit and energy in the allocation of jobs to available resources, without the need for such prior information. The approach is implemented in a software scheduler used to allocate real applications from the Princeton Application Repository for Shared-Memory Computers (PARSEC) benchmark suite to a number of hardware nodes realized with Odroid-XU3 boards. Experiments show that the proposed approach increases the profit earned by 40% while simultaneously reducing energy consumption by 20% when compared to a heuristic-based approach. We also present a network-aware server consolidation algorithm called Bandwidth-Constrained Consolidation (BCC), for HPC data centers which can address the under-utilization problem of the servers. Our experiments show that the BCC consolidation technique can reduce the power consumption of a data center by up-to 37%.
BibTeX
@article{mamun2020intra,
  author = {Sayed Ashraf Mamun and Alexander Gilday and Amit Kumar Singh and Amlan Ganguly and Geoff V. Merrett and Xiaohang Wang and Bashir M. Al‐Hashimi},
  title = {{Intra- and Inter-Server Smart Task Scheduling for Profit and Energy Optimization of HPC Data Centers}},
  journal = {Journal of Low Power Electronics and Applications},
  year = {2020},
  doi = {10.3390/jlpea10040032},
}
Figure