KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing

Sheng Qi, Zhiquan Zhang, Xuanzhe Liu, Xin Jin

TL;DR

KUBEDIRECT is a research breakthrough that allows Kubernetes to scale serverless functions up to 26.7x faster by bypassing the central API Server bottleneck. By enabling direct message passing between controllers while maintaining full compatibility with the existing ecosystem, it achieves the performance of "clean-slate" custom managers with only ~150 lines of code changes per controller.

What Problem Does This Paper Address?

Serverless computing (Function-as-a-Service or FaaS) lives and dies by its ability to scale instantly. When a burst of traffic hits, the platform must provision hundreds or thousands of function instances in milliseconds.

While Kubernetes is the industry standard for managing these containers, it wasn't built for this level of "burstiness." Its architecture relies on a state-centric control plane: when one controller (like an Autoscaler) wants to talk to another (like a Scheduler), it must write the state to a central API Server (backed by etcd), which then notifies the next controller. This "indirect write-notify" loop creates a massive bottleneck. Provisioning 100 pods can take tens of seconds in Kubernetes, even though the actual container startup takes less than a second.

Existing solutions usually involve throwing Kubernetes away and building a "clean-slate" manager from scratch. However, this loses access to the massive Kubernetes ecosystem of networking, monitoring, and security tools.

Methods

The authors introduce KUBEDIRECT, which optimizes the "narrow waist" of Kubernetes—the specific chain of controllers (Autoscaler → Deployment → ReplicaSet → Scheduler → Kubelet) responsible for scaling.

Key innovations include:

  • Direct Message Passing: Instead of going through the API Server, controllers send updates directly to their downstream neighbors via TCP links.
  • Dynamic Materialization: To keep messages tiny, KUBEDIRECT only sends the "delta" (the changes). For example, instead of sending a full 17KB Pod object, it might only send a 64-byte message containing a pointer to a template and a node assignment. The receiver then "materializes" the full object in memory.
  • Hierarchical Write-Back Cache: Since there is no longer a single central source of truth (the API Server), KUBEDIRECT treats the controller chain like a distributed cache. It uses a novel state management scheme to ensure that even if a controller crashes, the system eventually converges to the correct state.
  • Tombstones: A specialized mechanism to handle "downscaling" (deleting instances) asynchronously and idempotently, ensuring the system doesn't "revive" pods that were meant to be killed.

Datasets

The researchers evaluated KUBEDIRECT using:

  • Azure Functions Trace: A 30-minute representative slice of a real-world production trace from Microsoft Azure, containing 500 different functions and 168,000 invocations.
  • Synthetic Microbenchmarks: Scaling tests ranging from 100 to 800 pods and simulated large-scale clusters of up to 4,000 nodes.
  • Hardware: An 80-node physical cluster on CloudLab using Intel E5-2640v4 CPUs and 25 Gbps networking.

Key Insights and Ideas

  • The "Narrow Waist" Insight: Most FaaS platforms use the same core Kubernetes controllers. By optimizing just this small "waist," you can improve the performance of almost any FaaS platform (Knative, OpenFaaS, Fission) without a total rewrite.
  • Fungibility of Intermediate State: The authors realized that pods in the middle of being provisioned are "fungible"—they don't have side effects yet. If a controller fails, you don't need a persistent record in etcd; you can simply drop the ephemeral state and let the upstream controller recreate it.
  • Compatibility is King: Unlike previous research that required entirely new platforms, KUBEDIRECT requires only ~150 lines of code changes per controller, making it a "bolt-on" optimization rather than a replacement.

Key Results

  • Massive Speedup: KUBEDIRECT reduces function serving latency by 26.7× compared to standard Knative on Kubernetes.
  • Clean-Slate Performance: It achieves performance parity with Dirigent, a state-of-the-art "clean-slate" platform, while remaining inside the Kubernetes ecosystem.
  • Efficiency: Message sizes were reduced from an average of 17KB in standard Kubernetes to just 64B in KUBEDIRECT.
  • Scalability: In simulated tests, KUBEDIRECT successfully scaled 20,000 pods in just 30 seconds, a feat that would overwhelm a standard Kubernetes API Server.
  • Reduced Cold Starts: By speeding up the control plane, the system saw a 67% reduction in the number of "cold starts" during bursty workloads.

Limitations and Future Work

  • Manual Identification: The "narrow waist" must be manually identified and integrated. While common for FaaS, other types of workloads might have different controller chains.
  • Discovery Bottlenecks: While scaling is solved, the authors note that "discovery" (how functions find each other) still relies on the standard Kubernetes Service API, which could become a secondary bottleneck at extreme scales.
  • Annotation Requirement: Currently, users must add a specific annotation to their Deployment objects to opt-in to KUBEDIRECT management.
  • Future Direction: The authors suggest exploring how this direct-passing model could be applied to other "state-centric" systems beyond just Kubernetes, such as cluster managers for AI training or big data processing.

Summary updated

Abstract

FaaS platforms rely on cluster managers like Kubernetes for resource management. Kubernetes is popular due to its state-centric APIs that decouple the control plane into modular controllers. However, to scale out a burst of FaaS instances, message passing becomes the primary bottleneck as controllers have to exchange extensive state through the API Server. Existing solutions opt for a clean-slate redesign of cluster managers, but at the expense of compatibility with existing ecosystem and substantial engineering effort. We present KUBEDIRECT, a Kubernetes-based cluster manager for FaaS. We find that there exists a common narrow waist across FaaS platform that allows us to achieve both efficiency and external compatibility. Our insight is that the sequential structure of the narrow waist obviates the need for a single source of truth, allowing us to bypass the API Server and perform direct message passing for efficiency. However, our approach introduces a set of ephemeral states across controllers, making it challenging to enforce end-to-end semantics due to the absence of centralized coordination. KUBEDIRECT employs a novel state management scheme that leverages the narrow waist as a hierarchical write-back cache, ensuring consistency and convergence to the desired state. KUBEDIRECT can seamlessly integrate with Kubernetes, adding ~150 LoC per controller. Experiments show that KUBEDIRECT reduces serving latency by 26.7x over Knative, and has similar performance as the state-of-the-art clean-slate platform Dirigent.
BibTeX
@inproceedings{qi2026kubedirect,
  author = {Sheng Qi and Zhiquan Zhang and Xuanzhe Liu and Xin Jin},
  title = {{KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing}},
  booktitle = {NSDI},
  year = {2026},
}

Loading related content...

Figure