TL;DR
This paper introduces Naos, a novel JVM-based networking library that eliminates the need for traditional object serialization when transferring data between Java applications. By enabling direct, zero-copy RDMA communication of on-heap objects, Naos significantly boosts performance, offering substantial speedups in latency and throughput compared to existing serialization methods.
What Problem Does This Paper Address?
Modern distributed systems heavily rely on managed languages like Java and Scala. However, a significant performance bottleneck arises from the necessity of serializing and deserializing Java objects to transfer them over networks. This process involves expensive data transformations and memory copies, consuming substantial CPU and memory bandwidth. As network speeds increase, this serialization overhead becomes the dominant factor limiting performance, especially in large-scale data processing. Existing solutions, while improving serialization efficiency, still incur this fundamental overhead. The paper addresses the challenge of bypassing this serialization bottleneck entirely, enabling direct, high-performance communication for Java applications, particularly leveraging the capabilities of RDMA networks.
Methods
Naos introduces a serialization-free approach to Java networking. Its core innovation lies in allowing objects to be directly transferred from the sender's heap to the receiver's heap, bypassing the traditional serialization/deserialization pipeline.
Key techniques include:
- Direct On-Heap Object Transfer: Naos enables applications to send Java objects directly from their memory without converting them into byte streams. This is achieved through a custom JVM plugin and library.
- RDMA Integration: Naos leverages RDMA (Remote Direct Memory Access) for zero-copy data transfers, minimizing CPU involvement. It also supports traditional TCP.
- Object Graph Traversal (DFS): To identify all reachable objects, Naos employs a Depth-First Search (DFS) traversal, which is optimized for how Java objects are constructed.
- Pointer Recovery: Since objects are sent directly, their internal pointers are only valid in the sender's address space. Naos implements a sophisticated graph recovery algorithm on the receiver side to fix these pointers, making them valid in the receiver's heap. This involves fixing both class pointers and field pointers.
- Memory Management and GC Interaction: Naos carefully manages JVM memory to avoid issues with concurrent garbage collection. It uses object pinning and region-based pinning to ensure that objects being transmitted are not moved or deallocated by the GC.
- Pipelining: Naos supports pipelining of graph traversal and communication, allowing the receiver to start processing received data while the sender is still traversing the object graph. This is crucial for reducing latency in large object transfers.
- NaosIt: A specialized send call for Naos that compresses arrays with references, offering a trade-off between communication data size and receiver-side memory allocation.
Datasets
The paper does not explicitly mention training datasets. For evaluation, it uses:
- Synthetic Data Structures: Arrays of floats, Points (2D Euclidean points), and Pairs (integer and char array) of varying sizes to represent common data patterns.
- Real-World Graph Data: LiveJournal and Orkut datasets for evaluating PageRank, a representative data processing workload.
- Apache Dubbo: Used to demonstrate Naos's applicability in a real-world RPC framework.
- Key-Value Store (KVS): A custom KVS implementation populated with one million entries of 1 KiB each, benchmarked using YCSB workloads.
No new benchmarks were introduced; instead, existing workloads and frameworks were adapted to evaluate Naos.
Key Insights and Ideas
- Serialization is the Bottleneck: The paper strongly argues that as network speeds increase, traditional object serialization/deserialization becomes the primary performance limiter in distributed Java systems, consuming far more CPU time than network transfer itself.
- Zero-Copy RDMA is Key: Leveraging RDMA for direct, zero-copy transfers of on-heap objects is crucial for achieving significant performance gains by eliminating memory copies and CPU overhead.
- Pointer Recovery is Non-Trivial: Directly transferring objects necessitates a robust mechanism to fix internal pointers on the receiver's side, which is a complex but essential part of Naos.
- GC Interaction is Critical: Seamless integration with the JVM's garbage collector is vital to prevent data corruption or premature deallocation of objects during network transfer. Naos's approach to object pinning and region management is a key enabler.
- Pipelining Unlocks Latency Gains: Overlapping communication with computation (graph traversal and pointer fixing) through pipelining is essential for reducing end-to-end latency, especially for large and complex data structures.
- Trade-offs Exist: While Naos excels in many scenarios, its performance can be impacted by graph sparsity and the overhead of its pointer recovery mechanism compared to serializers that compress data more aggressively.
Key Results
- Latency Reduction: Naos RDMA offers significant latency reductions, achieving up to a 2.2x reduction for RPCs and 2x speedup for contiguous objects and moderately sparse graphs compared to serialization approaches. For large arrays of floats, Naos RDMA latency was at least 2x to 4.5x lower than Kryo and Java.
- Throughput Improvement: Naos RDMA achieved up to 1600 Kreq/sec for arrays of floats, a 2x speedup over existing serialization approaches. NaosIt provided further improvements by reducing network data size.
- CPU Cost Reduction: Naos RDMA shows at least a 2x reduction in receiver CPU time over Kryo and Java, as it avoids object construction and only fixes pointers.
- RDMA vs. TCP: Naos RDMA consistently outperforms Naos TCP, especially for larger data transfers, highlighting the benefits of RDMA's zero-copy capabilities.
- Dubbo Integration: Extending Apache Dubbo with Naos RDMA reduced RPC latency by at least 55%.
- YCSB Workloads: Naos RDMA achieved an average speedup of 11x over TCP-Kryo in a key-value store benchmark, attributed to lower CPU consumption.
Limitations and Future Work
- Data Format: Naos and Skyway's data format does not compress arrays with references as effectively as Kryo, leading to higher network costs for sparse graphs. The NaosIt variant addresses this to some extent.
- Sparsity Impact: For very sparse graphs, Naos's traversal can be slower than Kryo due to the overhead of its pointer recovery mechanism.
- JVM Compatibility: Naos relies on specific GC implementations (like Shenandoah) for its memory management features.
- SmartNIC Integration: The authors suggest that fixing pointers could be offloaded to SmartNICs, which could further improve performance by handling pointer recovery and class translation on the fly.
- Scatter-Gather I/O: Implementing scatter-gather capabilities for RDMA and TCP could enable building network messages from multiple buffers without intermediate buffering, potentially improving efficiency.
- TCP Optimization: While the focus is on RDMA, optimizing Naos TCP with pre-allocated buffers similar to Skyway could further improve its performance.
Summary updated