Abstract
Large multi-tenant production clusters often have to handle a variety of jobs and applications with a variety of complex resource usage characteristics. It is non-trivial and non-optimal to manually create placement rules for scheduling that would decide which applications should co-locate. In this paper, we present DeepPlace, a scheduler that learns to exploits various temporal resource usage patterns of applications using Deep Reinforcement Learning (Deep RL) to reduce resource competition across jobs running in the same machine while at the same time optimizing for overall cluster utilization.
Introduction
In cloud computing, "bin packing" is the fundamental challenge of allocating applications to server clusters efficiently. Traditionally, schedulers have relied on human-defined rules and static resource limits. However, as applications become more complex and their resource needs fluctuate dynamically, these static methods often lead to significant resource wastage or performance degradation due to contention.
DeepPlace, developed by researchers at Adobe Research and several Indian Institutes of Technology, proposes a shift from rigid rules to a self-learning scheduler. By utilizing Deep Reinforcement Learning (DRL), DeepPlace masters application placement by analyzing temporal resource usage patterns, effectively playing a high-stakes game of "Resource Tetris."
Problem Statement
Modern cluster managers like Kubernetes or YARN typically allocate resources based on conservative developer estimates. To avoid crashes during spikes, developers often request significantly more CPU and memory than required for the majority of the application's lifecycle. This results in "slack" resources that remain unused.
Furthermore, applications exhibit distinct "peaks" and "valleys" in resource consumption. If a scheduler co-locates multiple "peaky" applications on the same machine without considering their temporal alignment, they may spike simultaneously, leading to resource contention and performance lag. Manually crafting rules to avoid such overlaps at scale is increasingly impractical.
Methods
DeepPlace treats the scheduling problem as a reinforcement learning task where an agent interacts with the cluster environment to make placement decisions.
State Representation
A key innovation in DeepPlace is its state representation. Each machine's current and historical state is represented as a 2D matrix, where one axis denotes time (the last $k$ timesteps) and the other represents resource capacity. This "image-like" representation allows the model to capture and learn from the temporal patterns of active jobs.
Policy Network and Reward Function
The system employs a Deep Neural Network (DNN) trained using a modified REINFORCE algorithm. The network processes the cluster state and incoming job requests to output optimal placement actions.
To guide the learning process, the authors designed a multi-faceted reward function:
- Resource Contention: Penalizes overlapping peaks using cross-correlation.
- Over-utilization: Heavy penalties for exceeding machine capacity.
- Wait-time: Penalizes excessive queuing of jobs.
- Under-utilization: Encourages efficient packing to maximize cluster utility.
Datasets and Evaluation
The researchers validated DeepPlace using both real-world data and synthetic workloads:
- Production Data: Analysis of CPU usage from three Adobe production services confirmed the prevalence of periodic "peaks and valleys."
- Synthetic Workload: Jobs were generated via a Poisson process, modeled as "square waves" with varying periods and intensities to simulate diverse behaviors.
- Experimental Setup: The evaluation involved a cluster of 10 machines with 50 job types for training and 18 for testing.
Key Results
DeepPlace was benchmarked against the Tetris heuristic-based scheduler and a BestFit approach:
- Utilization: DeepPlace achieved a 68–100% increase in average utilization compared to Tetris.
- Efficiency: It handled identical workloads using fewer physical machines.
- Contention Management: Unlike BestFit, which often causes over-utilization spikes, DeepPlace maintained tight packing while keeping over-utilization near zero.
- Fragmentation: Resource fragmentation was reduced by 6–13%, preserving larger contiguous resource blocks for future large jobs.
Conclusion and Future Work
DeepPlace demonstrates that learning temporal "serendipitous dependencies"—pairing jobs with complementary idle times—is superior to static scheduling. While highly effective for batch and interactive tasks, future work will address scalability for larger action spaces in massive clusters and the "cold start" problem via historical log bootstrapping.
Summary updated