KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing

Sheng Qi, Zhiquan Zhang, Xuanzhe Liu, Xin Jin

Abstract

FaaS platforms rely on cluster managers like Kubernetes for resource management. Kubernetes is popular due to its state-centric APIs that decouple the control plane into modular controllers. However, to scale out a burst of FaaS instances, message passing becomes the primary bottleneck as controllers have to exchange extensive state through the API Server. Existing solutions opt for a clean-slate redesign of cluster managers, but at the expense of compatibility with existing ecosystem and substantial engineering effort. We present KUBEDIRECT, a Kubernetes-based cluster manager for FaaS. We find that there exists a common narrow waist across FaaS platform that allows us to achieve both efficiency and external compatibility. Our insight is that the sequential structure of the narrow waist obviates the need for a single source of truth, allowing us to bypass the API Server and perform direct message passing for efficiency. KUBEDIRECT employs a novel state management scheme that leverages the narrow waist as a hierarchical write-back cache, ensuring consistency and convergence to the desired state. KUBEDIRECT can seamlessly integrate with Kubernetes, adding ~150 LoC per controller. Experiments show that KUBEDIRECT reduces serving latency by 26.7x over Knative, and has similar performance as the state-of-the-art clean-slate platform Dirigent.

Introduction

Serverless computing (Function-as-a-Service or FaaS) lives and dies by its ability to scale instantly. When a burst of traffic hits, the platform must provision hundreds or thousands of function instances in milliseconds. While Kubernetes is the industry standard for managing these containers, it was not built for this level of "burstiness." Its architecture relies on a state-centric control plane: when one controller (like an Autoscaler) wants to talk to another (like a Scheduler), it must write the state to a central API Server (backed by etcd), which then notifies the next controller. This "indirect write-notify" loop creates a massive bottleneck. Provisioning 100 pods can take tens of seconds in Kubernetes, even though the actual container startup takes less than a second.

The "Narrow Waist" of FaaS

The authors introduce KUBEDIRECT, which optimizes the "narrow waist" of Kubernetes—the specific chain of controllers (Autoscaler → Deployment → ReplicaSet → Scheduler → Kubelet) responsible for scaling. Most FaaS platforms use the same core Kubernetes controllers. By optimizing just this small "waist," performance can be improved for almost any FaaS platform (Knative, OpenFaaS, Fission) without a total rewrite.

The "narrow waist" of FaaS platforms on Kubernetes and the associated API call complexity.

System Design and Innovations

KUBEDIRECT achieves high-performance scaling through several key innovations:

  • Direct Message Passing: Instead of going through the API Server, controllers send updates directly to their downstream neighbors via TCP links.
  • Dynamic Materialization: To keep messages tiny, KUBEDIRECT only sends "deltas." For example, instead of sending a full 17KB Pod object, it sends a 64-byte message containing a pointer to a template and a node assignment. The receiver then "materializes" the full object in memory.
  • Hierarchical Write-Back Cache: Since there is no longer a single central source of truth, KUBEDIRECT treats the controller chain like a distributed cache. It uses a novel state management scheme to ensure that even if a controller crashes, the system eventually converges to the correct state.
  • Fungibility of Intermediate State: Pods in the middle of being provisioned are "fungible"—they don't have side effects yet. If a controller fails, the system can drop the ephemeral state and let the upstream controller recreate it.

Evaluation and Results

The researchers evaluated KUBEDIRECT using a 30-minute representative slice of a real-world production trace from Microsoft Azure and synthetic microbenchmarks on an 80-node physical cluster.

Upscaling latency overhead in standard Kubernetes and the impact of keepalive settings on cold start counts.

Key results include:

  • Massive Speedup: KUBEDIRECT reduces function serving latency by 26.7× compared to standard Knative on Kubernetes.
  • Clean-Slate Performance: It achieves performance parity with Dirigent, a state-of-the-art "clean-slate" platform, while remaining inside the Kubernetes ecosystem.
  • Efficiency: Message sizes were reduced from an average of 17KB to just 64B.
  • Scalability: In simulated tests, KUBEDIRECT successfully scaled 20,000 pods in just 30 seconds.
  • Reduced Cold Starts: By speeding up the control plane, the system saw a 67% reduction in the number of "cold starts" during bursty workloads.

Conclusion and Future Work

KUBEDIRECT demonstrates that Kubernetes can be optimized for serverless scaling with minimal changes (~150 LoC per controller). While highly effective, the "narrow waist" must currently be manually identified. Future work could explore automating this process and addressing secondary bottlenecks like service discovery at extreme scales.

Summary updated

Abstract

FaaS platforms rely on cluster managers like Kubernetes for resource management. Kubernetes is popular due to its state-centric APIs that decouple the control plane into modular controllers. However, to scale out a burst of FaaS instances, message passing becomes the primary bottleneck as controllers have to exchange extensive state through the API Server. Existing solutions opt for a clean-slate redesign of cluster managers, but at the expense of compatibility with existing ecosystem and substantial engineering effort. We present KUBEDIRECT, a Kubernetes-based cluster manager for FaaS. We find that there exists a common narrow waist across FaaS platform that allows us to achieve both efficiency and external compatibility. Our insight is that the sequential structure of the narrow waist obviates the need for a single source of truth, allowing us to bypass the API Server and perform direct message passing for efficiency. However, our approach introduces a set of ephemeral states across controllers, making it challenging to enforce end-to-end semantics due to the absence of centralized coordination. KUBEDIRECT employs a novel state management scheme that leverages the narrow waist as a hierarchical write-back cache, ensuring consistency and convergence to the desired state. KUBEDIRECT can seamlessly integrate with Kubernetes, adding ~150 LoC per controller. Experiments show that KUBEDIRECT reduces serving latency by 26.7x over Knative, and has similar performance as the state-of-the-art clean-slate platform Dirigent.
BibTeX
@inproceedings{qi2026kubedirect,
  author = {Sheng Qi and Zhiquan Zhang and Xuanzhe Liu and Xin Jin},
  title = {{KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing}},
  booktitle = {NSDI},
  year = {2026},
}
Figure