TL;DR
RoCE BALBOA is an open-source, 100G-capable RDMA network stack designed specifically for FPGAs. Unlike "black-box" commercial NICs, BALBOA provides a highly customizable architecture that allows researchers and engineers to bake security features (like AES encryption) and compute offloads (like ML preprocessing) directly into the network datapath.
What Problem Does This Paper Address?
In modern data centers, the network is often the bottleneck for data-heavy applications like Machine Learning (ML). To solve this, the industry moved to Remote Direct Memory Access (RDMA), specifically RoCE v2 (RDMA over Converged Ethernet). RDMA is great because it bypasses the CPU, allowing one server to read/write another’s memory directly.
However, commercial RDMA hardware (ASIC-based NICs) is notoriously inflexible. This creates two major issues:
- Security Gaps: Standard RoCE v2 lacks native encryption and fine-grained access control, making it vulnerable to hijacking or side-channel attacks in public clouds.
- Fixed Functionality: You can’t easily add custom logic to a commercial NIC. If you want to preprocess data as it arrives from the wire before it hits the GPU, you’re usually out of luck or forced into high-latency workarounds.
Methods
The authors developed RoCE BALBOA using AMD Vitis High-Level Synthesis (HLS), making the hardware description more accessible than traditional RTL. The stack is modular and designed to run on FPGAs at 250 MHz with a 512-bit wide data bus, achieving a theoretical throughput of 128 Gbps (comfortably covering 100G line rates).
Key architectural features include:
- Modular Pipeline: Separate stages for IP, UDP, and InfiniBand header processing.
- Flexible Slots: Pre-defined "slots" for two types of enhancements:
- On-datapath (Serial): For tasks like AES encryption where data must be transformed before moving forward.
- Parallel-path: For tasks like Deep Packet Inspection (DPI) that analyze a copy of the data without delaying the main stream.
- Direct-to-GPU: Integration with the "Coyote" FPGA shell to allow Peer-to-Peer (P2P) DMA, moving processed data directly into GPU memory without touching the host CPU or RAM.
Datasets
While this is a systems paper rather than a pure ML paper, the authors used specific data to evaluate their custom offloads:
- DPI Training: To evaluate the ML-based Deep Packet Inspection, they trained a model on common big data payloads (CSVs, PNGs, TXTs) versus compiled malware executables.
- DLRM Operators: For the recommendation system use case, they implemented operators (Neg2Zero, Logarithm, Modulus) from Meta’s training toolkit to simulate real-world ML preprocessing pipelines.
Key Insights and Ideas
- Breaking the "Black Box": The most significant insight is that an open-source stack can match commercial performance while offering "service-enhancements." You no longer have to choose between 100G speeds and custom protocol logic.
- Latency Hiding: By using a "parallel-path" for ML-based packet inspection, BALBOA can perform complex security checks (inference) in just 44ns per beat, effectively hiding the analysis time within the existing network pipeline latency.
- The Power of Preprocessing: Moving ML data preparation (like normalization or clipping) to the NIC isn't just about speed; it's about "CPU tax" reduction. By doing this on the FPGA, they free up expensive CPU cores to do actual compute rather than just shuffling and cleaning data.
Key Results
- Performance Parity: BALBOA achieves 100G line-rate throughput and latency characteristics comparable to the industry-standard Mellanox ConnectX-5 NIC.
- Encryption Efficiency: While a 16-core server-grade CPU could only saturate a small fraction of a 100G link with AES encryption, BALBOA’s hardware-based AES implementation hit the full 100G line rate with negligible added latency.
- ML Preprocessing Speedup: In the Deep Learning Recommendation Model (DLRM) use case, the CPU-based preprocessing topped out at ~700 MB/s. BALBOA’s on-datapath implementation reached 8500 MB/s—a massive jump that allows the network to keep up with high-end GPUs.
Limitations and Future Work
- Scalability: The current implementation supports up to 500 Queue Pairs (QPs). While sufficient for many use cases, massive-scale cloud deployments might require further optimization of the state tables.
- Congestion Control: The paper uses a standard ACK-clocked flow control. The authors note that implementing more advanced algorithms like DCQCN or TIMELY is a priority for future versions to improve stability in congested networks.
- Future Features: The team plans to add line-rate compression/decompression to further save bandwidth and explore more complex "scale-out" applications.
Summary updated