TL;DR
NetRPC is a framework that makes high-performance In-Network Computation (INC) as easy to use as a standard gRPC call. By abstracting away complex hardware details and providing a familiar programming interface, it allows developers to achieve state-of-the-art performance for distributed applications while writing 95% less code.
What Problem Does This Paper Address?
In-Network Computation (INC) uses programmable network switches (like Barefoot Tofino) to perform data processing as packets fly through the wire. This can massively accelerate tasks like machine learning training, distributed consensus (Paxos), and caching.
However, there is a massive "usability gap." To use INC today, a developer must be an expert in network engineering and chip-specific languages like P4. They have to manually manage packet parsing, flow control, and hardware limitations (like the lack of floating-point math). Furthermore, most current INC solutions are "single-use"—you have to reboot the entire switch to change the application. NetRPC aims to turn the network into a shared, easy-to-program compute resource.
Methods
The authors designed NetRPC as a two-layer system that bridges the gap between high-level software and low-level hardware:
- RPCLayer: This provides a familiar gRPC-like interface. Developers define their services using
protobuf, but with special INC-Enabled Data Types (IEDTs) like floating-point arrays. Instead of writing P4 code, they provide a NetFilter—a simple JSON configuration that specifies which operations to perform. - INCLayer: This handles the "dirty work" of networking. It implements a reliable transport protocol specifically for INC, managing packet retransmission and congestion control (using ECN).
Key Technical Innovations:
* Reliable INC Primitives (RIPs): NetRPC offers five core operations (Map.addTo, Map.get, Map.clear, Stream.modify, and CntFwd) that can be composed to build complex applications.
* Software Fallback: If the switch runs out of memory or encounters an operation it can't handle (like an arithmetic overflow), the system automatically falls back to host-agent software. This ensures correctness without the developer needing to write "corner-case" logic.
* Automatic Data Parallelism: Large RPC calls are automatically broken into concurrent subtasks to fully saturate 100Gbps network links.
Datasets
The authors evaluated NetRPC across four distinct application types using standard benchmarks:
- Distributed ML Training (SyncAgtr): Evaluated using ImageNet with models like VGG, ResNet, and AlexNet.
- MapReduce (AsyncAgtr): Evaluated using a WordCount task on the Yelp dataset.
- Network Monitoring (KeyValue): Evaluated using CAIDA anonymized internet traces.
- Consensus (Agreement): Evaluated using a synthetic workload for the Paxos protocol.
Key Insights and Ideas
- Configuration over Programming: The most powerful insight is that 90% of INC use cases can be covered by a small set of primitives. By using a JSON-based "filter" instead of a full programming language, the barrier to entry drops significantly.
- The Idempotency Problem: Retransmitting a packet in standard networking is fine, but in INC, it might cause a value to be added twice (e.g.,
x += 1). NetRPC solves this with a clever per-flow bit array on the switch that tracks whether a specific packet has already been processed. - Memory as a Cache: Since switch memory is tiny compared to host RAM, NetRPC treats switch registers as a cache for the most "hot" keys, using a periodic counting-based LRU (Least Recently Used) policy to swap data in and out.
- Multi-Tenancy: Unlike prior work, NetRPC allows multiple different INC applications to run on the same switch simultaneously without needing a reboot.
Key Results
- Drastic Code Reduction: NetRPC reduced the human-written lines of code (LoC) by over 97% compared to specialized INC solutions. For example, a distributed training setup that previously required ~8,700 lines of code was reduced to just ~186 lines.
- Performance Parity: Despite being much easier to use, NetRPC achieves 97% to 100% of the performance of state-of-the-art, handcrafted INC systems like ATP.
- Throughput & Latency: NetRPC achieved ~50-73 Gbps of goodput and reduced consensus (Paxos) latency from 92µs (software-only) to just 20µs.
- Robustness: The system maintains high performance even with 1% packet loss, outperforming specialized systems like SwitchML in degraded network conditions.
Limitations and Future Work
- Logic Complexity: Because NetRPC relies on a fixed set of primitives to maintain simplicity, it cannot easily implement applications requiring complex branching or looping logic directly on the switch.
- Advanced Scheduling: The current version uses a simple First-Come-First-Served (FCFS) policy for switch memory. The authors suggest that more sophisticated scheduling will be needed as more applications share the same data plane.
- Topology Scaling: The evaluation was performed on a two-switch testbed. Future work is needed to explore how NetRPC handles more complex, oversubscribed data center topologies.
Summary updated