Kernel Bypass and Remote Direct Memory Access

364 concepts226 papers2 proposals

In traditional operating systems, when a network packet arrives at a Network Interface Card (NIC), it triggers a hardware interrupt. The CPU halts its current task, switches context from user space to kernel space, copies the packet from the NIC's ring buffer into kernel memory, processes the network stack (such as TCP/IP), and finally copies the data a second time into the application's user-space memory.

While this protective boundary between applications and hardware worked well for decades, modern high-speed networks have rendered it a massive performance bottleneck. As noted by Timo Schneider et al. in FPsPIN: An FPGA-based Open-Hardware Research Platform for Processing in the Network, at network speeds of 400 Gbps and beyond, a new packet arrives faster than the time it takes a CPU to access a single line of L3 cache. The CPU spends all its computational cycles managing interrupts and copying data across protection domains, leaving little room for actual application logic.

To overcome this "utilization wall," systems researchers have developed a suite of techniques known as Kernel Bypass and Remote Direct Memory Access (RDMA). The core intuition behind these technologies is simple: eliminate the operating system kernel from the data path entirely, transforming data movement from a software scheduling problem into a pure hardware routing task.

Kernel Bypass: Bringing Hardware to User Space

Kernel bypass works by mapping a hardware device's memory-mapped I/O (MMIO) registers directly into an application's user-space virtual memory address space. Once this mapping is established, the application can interact with the hardware directly, bypassing the operating system kernel entirely.

Instead of relying on expensive, interrupt-driven notifications when new data arrives, kernel-bypass frameworks typically employ a polling-based driver model. The application dedicates specific CPU cores to continuously scan a lock-free ring buffer in user space for new descriptors. As explained by Yan Jiang et al. in Design and Implementation of IPsec VPN IoT Gateway System in National Secret Algorithm, capturing network packets directly into lock-free circular buffers eliminates the CPU overhead of context switching and protection-domain crossings.

This approach dramatically lowers latency and jitter. Work by Charalampos Stylianopoulos et al. in On the performance of commodity hardware for low latency and low jitter packet processing demonstrates that moving packet handling directly to user space allows commodity hardware to achieve the ultra-low latency required by time-critical industrial networks. For instance, Armir Bujari et al. leverage kernel-bypass networking in A Layered Architecture Enabling Metaverse Applications in Smart Manufacturing Environments to stream high-resolution 4K and 8K video at frame rates exceeding 100 FPS, eliminating the latency spikes that cause motion sickness in augmented reality interfaces.

This paradigm extends beyond network packets to storage architectures. In SpanDB: A Fast, Cost-Effective LSM-tree Based KV Store on Hybrid Storage, Hao Chen et al. utilize the Storage Performance Development Kit (SPDK) to grant user-space applications direct access to NVMe SSDs. By bypassing the kernel's file system and block layers, write-ahead logging operations can saturate the physical limits of modern flash storage. Similarly, Wei Cao 0006 et al. demonstrate in PolarFS: An Ultra-low Latency and Failure Resilient Distributed File System for Shared Storage Cloud Database that combining user-space I/O polling with Direct Memory Access (DMA) prevents the operating system from throttling high-throughput cloud databases.

Remote Direct Memory Access (RDMA) and One-Sided Communication

While kernel bypass solves the data-transfer bottleneck within a single machine, distributed systems require moving data across different physical servers. Traditional network transfers require the CPUs on both the sending and receiving machines to actively participate in parsing network protocols.

Remote Direct Memory Access (RDMA) solves this by offloading the entire network stack onto specialized hardware (such as InfiniBand or RoCE-enabled NICs). As described by Jun-ya Kato et al. in Open Source Software Efforts to Transform IoT/AI Services, RDMA allows a local network card to read or write directly to the memory of a remote computer without involving the operating system or CPU of either machine.

Traditional Network Stack:
[User Space]  ---> [Kernel Space (TCP/IP)] ---> [NIC] === (Network) ===> [NIC] ---> [Kernel Space] ---> [User Space]
   (Copy 1)             (Interrupt)                                                   (Interrupt)         (Copy 2)

RDMA (Kernel Bypass):
[User Space Memory] -------------------------> [RDMA NIC] === (Network) ===> [RDMA NIC] -------------------------> [User Space Memory]
                      (Direct Hardware DMA)                                              (Direct Hardware DMA)

This architecture enables a paradigm known as one-sided communication. As Jorge González‐Domínguez explains in Parallel and Scalable Short-Read Alignment on Multi-Core Clusters Using UPC++, one-sided primitives allow a compute node to manipulate a remote node's memory space without requiring the remote processor to execute a matching receive command or coordinate synchronization. This decouples the execution paths of distributed nodes, drastically reducing communication overhead.

To optimize synchronization further, researchers minimize payload overhead. Torsten Hoefler demonstrates in Evaluation of publicly available Barrier-Algorithms and Improvement of the Barrier-Operation for large-scale Cluster-Systems with special Attention on InfiniBand Networks that embedding small synchronization messages directly within an RDMA write descriptor allows systems to bypass the receiver's CPU entirely. This technique avoids separate memory copy operations for minimal payloads, enabling constant-time barrier operations across massive clusters.

By shifting the performance bottleneck from software overhead to raw hardware wire speeds, RDMA allows traditional distributed algorithms to scale linearly. For example, Claude Barthels et al. show in Strong consistency is not hard to get: Two-Phase Locking and Two-Phase Commit on Thousands of Cores that classic database concurrency protocols like Two-Phase Locking (2PL) and Two-Phase Commit (2PC)—long thought to be unscalable due to network delays—can scale efficiently to thousands of cores when backed by low-latency RDMA fabrics.

Peer-to-Peer Transfers: Bypassing Host Memory

In modern accelerated computing workloads, such as Large Language Model (LLM) training and inference, data is processed on hardware accelerators like GPUs rather than the host CPU. In a conventional system, moving data from a high-speed network card or an NVMe SSD to a GPU requires staging the data through host system RAM, forcing the CPU to orchestrate the transfer across the PCIe bus.

To eliminate this intermediate step, modern architectures utilize peer-to-peer DMA paths, such as GPUDirect RDMA (GDR) and GPUDirect Storage (GDS). These protocols allow peripheral devices to communicate directly with each other over the PCIe bus or custom interconnects, bypassing host DRAM and the host CPU entirely.

Host-Mediated Transfer (Slow):
[NVMe SSD / NIC] ---> [Host System RAM] ---> [Host CPU] ---> [Host System RAM] ---> [GPU Memory]

Peer-to-Peer GPUDirect Transfer (Fast):
[NVMe SSD / NIC] ======================= (Direct PCIe Path) =======================> [GPU Memory]

In Understanding the Benefits of Hardware-Accelerated Communication in Model-Serving Applications, Walid A. Hanafy et al. demonstrate that allowing network interface cards to write data directly into a remote GPU's memory slashes end-to-end model-serving latency by 15% to 50% in multi-stage edge pipelines. This direct communication path prevents the host processor from becoming a bottleneck in distributed multi-GPU environments, an effect analyzed by Ang Li et al. in Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect.

This mechanism is equally critical for balancing I/O and compute when working with large datasets. Jing Li et al. show in HippogriffDB: Balancing I/O and GPU Bandwidth in Big Data Analytics that direct peer-to-peer data transfers between storage devices and GPUs are essential for unlocking the full potential of high-speed SSDs during analytical database queries.

In memory-constrained deep learning workloads, this architecture allows secondary storage to act as an efficient extension of accelerator memory. For instance, Ziqi Yuan et al. introduce a lifetime-aware tensor offloading framework in Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage, using GPUDirect Storage to stream inactive model weights directly to NVMe drives. This approach reduces host CPU utilization by over 97% and eliminates host-mediated offloading bottlenecks.

Similarly, Inho Jeong et al. demonstrate in HiFC: High-efficiency Flash-based KV Cache Swapping for Scaling LLM Inference that swapping LLM Key-Value (KV) caches directly between GPU memory and flash storage via GDS mitigates PCIe bandwidth bottlenecks. This direct-to-storage architecture enables systems to support massive context windows that would otherwise be cost-prohibitive to maintain in volatile memory.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_remote_direct_memory_access_6b0b3198,
  title = {remote direct memory access},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/remote-direct-memory-access/rdr_6b0b31987cb743c38330a7a30238c950},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_remote_direct_memory_access_6b0b3198_v1,
  title = {remote direct memory access},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/remote-direct-memory-access/rdr_6b0b31987cb743c38330a7a30238c950/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake