Abstract
Warehouse-scale cloud datacenters co-locate workloads with different and often complementary characteristics for improved resource utilization. To better understand the challenges in managing such intricate, heterogeneous workloads while providing quality-assured resource orchestration and user experience, we analyze Alibaba's co-located workload trace, the first publicly available dataset with precise information about the category of each job. Two types of workload—long-running, user-facing, containerized production jobs, and transient, highly dynamic, non-containerized, and non-production batch jobs—are running on a shared cluster of 1313 machines. Our multifaceted analysis reveals insights that we believe are useful for system designers and IT practitioners working on cluster management systems.
Introduction
Warehouse-scale datacenters represent significant capital investments, yet they frequently suffer from low resource utilization. Industry reports indicate that average CPU utilization often hovers between 20% and 40% for major providers like Google, with some servers operating as low as 7%. To optimize efficiency and reduce costs, providers employ "co-location," running diverse workloads on the same physical hardware.
However, co-location introduces complex management challenges. Long-running production services (e.g., web servers) require high availability and low latency, while transient batch jobs (e.g., data processing) demand high throughput. Balancing these competing needs without performance degradation—often caused by "noisy neighbor" effects—is a critical task for modern cluster management.
System Architecture and Methodology
This study analyzes the 2017 Alibaba Cluster Trace, which covers 1,313 machines over a 24-hour period. A unique feature of this dataset is the explicit categorization of jobs into "long-running containerized production" and "non-production batch" workloads.
The cluster utilizes a two-level scheduling architecture. Sigma manages production containers, while Fuxi handles batch jobs. A Level-0 agent on each machine coordinates resource allocation, yet the analysis reveals that these two scheduling systems often operate with limited communication, leading to potential hotspots and performance interference.
Workload Insights: The Scavenging Relationship
The analysis highlights a "complementary" but complex relationship between workload types:
- Idle Production Containers: Approximately 60% of production containers use less than 1% of their allocated CPU. Despite this low CPU usage, they remain "memory-hungry," maintaining high memory reservations that stabilize cluster-wide memory utilization.
- Elastic Scavenging: Batch jobs act as opportunistic "fillers," scavenging the idle CPU cycles reserved by production containers. Interestingly, about 30% of batch tasks exceed their official CPU requests, effectively "stealing" unused capacity from their production neighbors.
- Stragglers and Starvation: The study distinguishes between Stragglers (tasks delayed during execution due to resource contention or node issues) and Starvers (tasks delayed in the scheduling queue). Approximately 7% of tasks were identified as stragglers, taking five times longer than average. Some batch tasks experienced extreme starvation, waiting over six hours to begin execution.
Cluster Utilization and Performance
Despite co-location efforts, the cluster remains underutilized in terms of CPU.
As shown in the resource usage analysis, the cluster spends 80% of its time at only 10–30% CPU utilization. Memory usage is significantly higher and more stable, generally exceeding 50%. The data suggests that production jobs are heavily over-provisioned for CPU to accommodate potential spikes, creating the "ample opportunity space" that batch jobs exploit. However, when CPU utilization exceeds 40% or memory exceeds 80%, performance metrics like Cycles Per Instruction (CPI) spike, indicating that co-location efficiency degrades at higher loads.
Conclusion and Future Directions
The Alibaba case study underscores the benefits and pitfalls of workload co-location. While batch jobs successfully scavenge idle resources, the lack of coordination between independent schedulers leads to performance interference and straggler issues.
Future research should focus on:
- Integrated Scheduling: Developing global controllers that holistically manage both production and batch workloads to minimize interference.
- Predictive Modeling: Utilizing machine learning to better forecast container resource footprints, allowing for safer and tighter packing.
- Hardware Heterogeneity: Investigating how these dynamics evolve with the inclusion of specialized accelerators like GPUs.
Summary updated