RoCE BALBOA: Service-enhanced Data Center RDMA for SmartNICs

Maximilian Jakob Heer, Benjamin Ramhorst, Yu Zhu, Luhao Liu, Zhiyi Hu, Jonas Dann, Gustavo Alonso

TL;DR

RoCE BALBOA is an open-source, 100G-capable RDMA network stack designed specifically for FPGAs. Unlike "black-box" commercial NICs, BALBOA provides a highly customizable architecture that allows researchers and engineers to bake security features (like AES encryption) and compute offloads (like ML preprocessing) directly into the network datapath.

What Problem Does This Paper Address?

In modern data centers, the network is often the bottleneck for data-heavy applications like Machine Learning (ML). To solve this, the industry moved to Remote Direct Memory Access (RDMA), specifically RoCE v2 (RDMA over Converged Ethernet). RDMA is great because it bypasses the CPU, allowing one server to read/write another’s memory directly.

However, commercial RDMA hardware (ASIC-based NICs) is notoriously inflexible. This creates two major issues:

  1. Security Gaps: Standard RoCE v2 lacks native encryption and fine-grained access control, making it vulnerable to hijacking or side-channel attacks in public clouds.
  2. Fixed Functionality: You can’t easily add custom logic to a commercial NIC. If you want to preprocess data as it arrives from the wire before it hits the GPU, you’re usually out of luck or forced into high-latency workarounds.

Methods

The authors developed RoCE BALBOA using AMD Vitis High-Level Synthesis (HLS), making the hardware description more accessible than traditional RTL. The stack is modular and designed to run on FPGAs at 250 MHz with a 512-bit wide data bus, achieving a theoretical throughput of 128 Gbps (comfortably covering 100G line rates).

Key architectural features include:

  • Modular Pipeline: Separate stages for IP, UDP, and InfiniBand header processing.
  • Flexible Slots: Pre-defined "slots" for two types of enhancements:
  • On-datapath (Serial): For tasks like AES encryption where data must be transformed before moving forward.
  • Parallel-path: For tasks like Deep Packet Inspection (DPI) that analyze a copy of the data without delaying the main stream.
  • Direct-to-GPU: Integration with the "Coyote" FPGA shell to allow Peer-to-Peer (P2P) DMA, moving processed data directly into GPU memory without touching the host CPU or RAM.

Datasets

While this is a systems paper rather than a pure ML paper, the authors used specific data to evaluate their custom offloads:

  • DPI Training: To evaluate the ML-based Deep Packet Inspection, they trained a model on common big data payloads (CSVs, PNGs, TXTs) versus compiled malware executables.
  • DLRM Operators: For the recommendation system use case, they implemented operators (Neg2Zero, Logarithm, Modulus) from Meta’s training toolkit to simulate real-world ML preprocessing pipelines.

Key Insights and Ideas

  • Breaking the "Black Box": The most significant insight is that an open-source stack can match commercial performance while offering "service-enhancements." You no longer have to choose between 100G speeds and custom protocol logic.
  • Latency Hiding: By using a "parallel-path" for ML-based packet inspection, BALBOA can perform complex security checks (inference) in just 44ns per beat, effectively hiding the analysis time within the existing network pipeline latency.
  • The Power of Preprocessing: Moving ML data preparation (like normalization or clipping) to the NIC isn't just about speed; it's about "CPU tax" reduction. By doing this on the FPGA, they free up expensive CPU cores to do actual compute rather than just shuffling and cleaning data.

Key Results

  • Performance Parity: BALBOA achieves 100G line-rate throughput and latency characteristics comparable to the industry-standard Mellanox ConnectX-5 NIC.
  • Encryption Efficiency: While a 16-core server-grade CPU could only saturate a small fraction of a 100G link with AES encryption, BALBOA’s hardware-based AES implementation hit the full 100G line rate with negligible added latency.
  • ML Preprocessing Speedup: In the Deep Learning Recommendation Model (DLRM) use case, the CPU-based preprocessing topped out at ~700 MB/s. BALBOA’s on-datapath implementation reached 8500 MB/s—a massive jump that allows the network to keep up with high-end GPUs.

Limitations and Future Work

  • Scalability: The current implementation supports up to 500 Queue Pairs (QPs). While sufficient for many use cases, massive-scale cloud deployments might require further optimization of the state tables.
  • Congestion Control: The paper uses a standard ACK-clocked flow control. The authors note that implementing more advanced algorithms like DCQCN or TIMELY is a priority for future versions to improve stability in congested networks.
  • Future Features: The team plans to add line-rate compression/decompression to further save bandwidth and explore more complex "scale-out" applications.

Summary updated

Abstract

Data-intensive applications in data centers, especially machine learning (ML), have made the network a bottleneck, which in turn has motivated the development of more efficient network protocols and infrastructure. For instance, remote direct memory access (RDMA) has become the standard protocol for data transport in the cloud as it minimizes data copies and reduces CPU-utilization via host-bypassing. Similarly, an increasing amount of network functions and infrastructure have moved to accelerators, SmartNICs, and in-network computing to bypass the CPU. In this paper we explore the implementation and deployment of RoCE BALBOA, an open-source, RoCE v2-compatible, scalable up to hundreds of queue-pairs, and 100G-capable RDMA-stack that can be used as the basis for building accelerators and smartNICs. RoCE BALBOA is customizable, opening up a design space and offering a degree of adaptability not available in commercial products. We have deployed BALBOA in a cluster using FPGAs and show that it has latency and performance characteristics comparable to commercial NICs. We demonstrate its potential by exploring two classes of use cases. One involves enhancements to the protocol for infrastructure purposes (encryption, deep packet inspection using ML). The other showcases the ability to perform line-rate compute offloads with deep pipelines by implementing commercial data preprocessing pipelines for recommender systems that process the data as it arrives from the network before transferring it directly to the GPU. These examples demonstrate how BALBOA enables the exploration and development of SmartNICs and accelerators operating on network data streams.
BibTeX
@misc{heer2025roce,
  author = {Maximilian Jakob Heer and Benjamin Ramhorst and Yu Zhu and Luhao Liu and Zhiyi Hu and Jonas Dann and Gustavo Alonso},
  title = {{RoCE BALBOA: Service-enhanced Data Center RDMA for SmartNICs}},
  year = {2025},
  eprint = {2507.20412},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.20412},
}
Figure