KRCORE: A Microsecond-scale RDMA Control Plane for Elastic Computing

Xingda Wei, Fangming Lu, Rong Chen, Haibo Chen

Abstract

Remote Direct Memory Access (RDMA) provides the low-latency networking required by modern data centers. However, while its data path operates at microsecond scales, its control path—responsible for connection setup—is significantly slower, often taking milliseconds. This discrepancy creates a bottleneck for "elastic" applications like serverless functions. We present KRCORE, a high-performance RDMA library that reduces connection setup times from ~15ms to under 10µs. By virtualizing kernel-space connections and leveraging Dynamic Connected Transport (DCT), KRCORE enables instantaneous scaling for cloud-native workloads.

The Problem: Millisecond Control in a Microsecond World

In elastic computing environments, tasks are frequently launched and destroyed to handle load spikes. RDMA's traditional connection setup (the control path) involves complex handshakes and hardware initialization that can take up to 15ms. For a task that only needs to send a few kilobytes of data, the network setup time can be 15,000 times longer than the actual transfer.

Figure 1: RDMA control path overhead. In serverless environments (right), the black bars representing RDMA control time dominate the total execution time.

As illustrated in Figure 1, this overhead is particularly severe in serverless platforms, where it throttles the system's ability to scale up nearly instantaneously.

Methods: Virtualization and Hybrid Pooling

KRCORE is implemented as a Linux kernel module that addresses these bottlenecks through three key design principles:

  1. DCT Virtualization: KRCORE virtualizes Dynamic Connected Transport (DCT), allowing a single connection point to communicate with multiple hosts on-the-fly. It maintains a pool of pre-initialized Virtual Queue Pairs (VQPs) in the kernel to bypass slow hardware initialization.
  2. RDMA-based Meta Server: Instead of using CPU-intensive RPC calls for connection handshakes, KRCORE uses one-sided RDMA READ operations to fetch metadata from global servers, improving throughput by over 11x.
  3. Hybrid QP Pool: To balance the fast setup of DCT with the high peak throughput of standard Reliable Connected (RC) pairs, KRCORE employs a hybrid approach. It uses DCT for immediate communication and establishes an RC connection in the background for long-term traffic.
Figure 6: The Hybrid QP Pool mechanism. KRCORE provides instant connectivity via DCT while transparently establishing high-performance RC connections in the background.

Key Results

Evaluations using systems like RACE Hashing and the Fn serverless platform demonstrate KRCORE's effectiveness:

  • Connection Latency: KRCORE establishes connections in ~10µs, a massive improvement over standard Verbs (15.7ms) and previous state-of-the-art solutions like LITE (2ms).
  • Scalability: It supports up to 22 million connections per second, whereas standard methods bottleneck at around 712.
  • Application Impact: In serverless workloads, KRCORE reduced data transfer latency by 99% for small payloads and worker bootstrap time by 83% during load spikes.
  • Memory Efficiency: KRCORE supports 5,000 connections with just 6.3MB of memory, compared to 780MB for LITE.

Conclusion

KRCORE successfully shrinks RDMA connection times to the microsecond scale, removing a critical barrier for elastic data center applications. While operating in the kernel adds some data path overhead, the trade-off is highly beneficial for the short-lived, high-frequency connection patterns characteristic of modern cloud computing.

Summary updated

BibTeX
@inproceedings{wei2022krcore,
  author = {Xingda Wei and Fangming Lu and Rong Chen and Haibo Chen},
  title = {{KRCORE: A Microsecond-scale RDMA Control Plane for Elastic Computing}},
  booktitle = {USENIX ATC},
  year = {2022},
  url = {https://openreview.net/forum?id=FIPcdwYiMH},
}
Figure