Characterizing Co-located Datacenter Workloads: An Alibaba Case Study

Yue Cheng, Zheng Chai, Ali Anwar

TL;DR

This paper provides a deep-dive analysis of Alibaba’s 2017 cluster trace, revealing how the tech giant co-locates long-running production services with transient batch jobs to fight low resource utilization. The study highlights a "complementary" relationship where idle resources from over-provisioned production containers are scavenged by batch jobs, though this comes at the cost of performance interference and persistent "straggler" issues.

What Problem Does This Paper Address?

Warehouse-scale datacenters are notoriously expensive and often suffer from dismal resource utilization—Google has reported averages between 20–40%, while some Amazon servers sit at just 7%. To save money, providers "co-locate" different types of work on the same machines.

However, managing this is a tightrope walk. You have long-running production jobs (like web services) that need high availability and low latency, and batch jobs (like data processing) that are high-throughput but less time-sensitive. The challenge lies in understanding how these "noisy neighbors" interact, how to schedule them without causing performance meltdowns, and why old problems like "stragglers" (tasks that take much longer than their peers) still haunt modern systems.

Methods

The authors perform a multifaceted characterization study. Instead of proposing a new algorithm, they analyze a massive real-world dataset to extract patterns in:

  • Resource Reservation vs. Usage: Comparing what users ask for versus what they actually use.
  • Temporal Dynamics: How load fluctuates over a 24-hour period.
  • Scheduling Efficiency: Analyzing how Alibaba’s two-level scheduler (Sigma for containers, Fuxi for batch) manages the cluster.
  • Performance Interference: Using hardware metrics like CPI (Cycles Per Instruction) and MPKI (Memory accesses Per Kilo-Instruction) to measure how "crowded" a machine feels to the software running on it.

Datasets

The study uses the 2017 Alibaba Cluster Trace.

  • Scope: 1,313 machines over a 24-hour period.
  • Uniqueness: Unlike the famous Google traces, the Alibaba trace explicitly categorizes jobs into "long-running containerized production" and "non-production batch." This allows researchers to see exactly how these two distinct species of workload interact on the same hardware.

Key Insights and Ideas

  • The "Idle" Production Container: A staggering ~60% of production containers are essentially inactive, using less than 1% of their allocated CPU. However, they are "memory-hungry," keeping memory utilization much higher than CPU utilization across the cluster.
  • Elastic Scavenging: Batch jobs act as the cluster's "fillers." They elastically over-commit to resources that production jobs have reserved but aren't using. About 30% of batch tasks actually use more CPU than they officially requested, "stealing" the idle cycles from production containers.
  • Stragglers vs. Starvers: The authors distinguish between Stragglers (tasks that start on time but finish late due to bad luck or poor nodes) and Starvers (tasks that wait a long time to even start). They found that 7% of tasks are stragglers taking 5x longer than average, often due to Fuxi’s speculative execution failing or resource contention.
  • Disconnected Schedulers: The analysis suggests that the two scheduling systems (Sigma and Fuxi) don't communicate enough. Fuxi often treats the cluster as a homogeneous pool, ignoring the "hotspots" created by production containers, which leads to avoidable performance interference.

Key Results

  • Utilization: The cluster spends 80% of its time at only 10–30% CPU utilization, despite the co-location efforts. Memory usage is more stable, generally staying above 50%.
  • Over-provisioning: Production jobs are heavily over-provisioned for CPU to handle potential spikes, creating the "ample opportunity space" for batch jobs.
  • Performance Hit: When CPU utilization crosses 40% or memory crosses 80%, performance metrics (CPI and MPKI) spike significantly, indicating that co-location starts to hurt efficiency at high loads.
  • Starvation: Some batch tasks suffered extreme delays, with the longest "starvation delay" reaching over 6 hours (23,379 seconds).

Limitations and Future Work

The authors acknowledge that some findings might be specific to Alibaba’s unique infrastructure. Furthermore, the 24-hour trace is a relatively small window into long-term operations.

Future Directions: 1. Integrated Scheduling: Moving away from independent schedulers toward a "co-location-optimized" global controller that understands the nuances of both workload types. 2. Machine Learning for Prediction: Using ML to better predict the "footprint" and "spikiness" of containers to allow for tighter, safer packing of jobs. 3. Heterogeneous Accelerators: Exploring how these dynamics change when GPUs and other specialized hardware are added to the mix.

Summary updated

Abstract

Warehouse-scale cloud datacenters co-locate workloads with different and often complementary characteristics for improved resource utilization. To better understand the challenges in managing such intricate, heterogeneous workloads while providing quality-assured resource orchestration and user experience, we analyze Alibaba's co-located workload trace, the first publicly available dataset with precise information about the category of each job. Two types of workload---long-running, user-facing, containerized production jobs, and transient, highly dynamic, non-containerized, and non-production batch jobs---are running on a shared cluster of 1313 machines. Our multifaceted analysis reveals insights that we believe are useful for system designers and IT practitioners working on cluster management systems.
BibTeX
@inproceedings{cheng2018characterizing,
  author = {Yue Cheng and Zheng Chai and Ali Anwar},
  title = {{Characterizing Co-located Datacenter Workloads: An Alibaba Case Study}},
  booktitle = {APSys},
  year = {2018},
  doi = {10.1145/3265723.3265742},
}

Loading related content...

Figure