TL;DR
This paper introduces Platinum, a novel garbage collector designed to significantly reduce tail latency in interactive services without a proportional increase in CPU usage. By leveraging hardware features like Memory Protection Keys (MPK) and a novel heap partitioning strategy, Platinum achieves isolated execution between garbage collection threads and application threads, leading to substantial performance improvements.
What Problem Does This Paper Address?
Modern interactive services, often built on managed languages like Java, face a critical challenge: garbage collection (GC). While GC automates memory management, its "stop-the-world" (STW) pauses can introduce significant latency, especially in latency-sensitive interactive services where requests are expected to complete in milliseconds. Existing solutions to reduce STW pauses, like partially-concurrent or mostly-concurrent collectors, often come with a trade-off: either more frequent but shorter pauses (partially-concurrent) or reduced pauses at the cost of substantially higher CPU consumption (mostly-concurrent). This increased CPU usage can degrade application performance under heavy loads, creating a dilemma for developers aiming for both low tail latency and efficient resource utilization.
Methods
Platinum tackles this problem with a multi-pronged approach:
- CPU-Efficient Concurrent GC: Platinum aims to reduce tail latency while keeping CPU consumption moderate.
- Isolated Execution Environment: It constructs an isolated execution environment for concurrent mutators (application threads) to minimize interference with GC threads.
- Hardware-Assisted Barrier Elimination: It leverages Intel Memory Protection Keys (MPK) to eliminate the software overhead associated with traditional concurrent collectors.
- Heap Partitioning: Platinum partitions the Java heap into three areas during GC: a "collection area" for GC threads, a "pinned area" for objects likely to be modified by mutators, and an "allocation area" for new object allocations. This partitioning allows GC threads and mutators to focus on different parts of the heap, reducing synchronization needs.
- Leveraging Idle Cores: When GC is not active, Platinum allows mutators to run on any available CPU cores. When GC is triggered, mutators are constrained to use only the "idle cores" not occupied by GC threads, ensuring efficient use of all available resources.
- Hardware Transactional Memory (RTM): For handling "dual-copy" problems (where an object is copied and then modified), Platinum uses RTM to ensure atomic updates to both copies, providing strong consistency with minimal overhead.
Datasets
The paper evaluates Platinum using:
- Production traces from Alibaba: These traces represent real-world interactive service workloads, specifically for the coupon service.
- Standard benchmarks: SpecJBB2015 (simulating an online supermarket) and Cassandra (a key-value store) were used to evaluate performance across different application types.
Key Insights and Ideas
- Skewed Memory Write Behavior: A crucial observation is that in session-based interactive services, mutators tend to update objects within a very small "working set." This skewed behavior is a key enabler for Platinum's heap partitioning strategy.
- The "Dual-Copy" Problem: When objects are copied during GC, and mutators continue to modify them, maintaining consistency between the old and new copies is challenging. Platinum's solution using RTM is a novel hardware-assisted approach to this problem.
- MPK as a Performance Enabler: The paper highlights how MPK, typically used for security, can be effectively repurposed to improve GC performance by enabling fine-grained isolation and reducing software barrier overhead.
- Trade-off Between Latency and CPU: The study reinforces that there's a fundamental trade-off between reducing GC pause times and CPU efficiency. Platinum aims to find a sweet spot by intelligently managing these resources.
Key Results
- Significant Tail Latency Reduction: Platinum reduced tail latency by as much as 79.3% for real-world interactive services compared to other collectors.
- Moderate CPU Consumption: Despite its performance gains, Platinum maintained moderate CPU utilization, often lower than or comparable to other advanced collectors like G1 and Shenandoah, especially under high throughput.
- SpecJBB2015 Performance: Platinum outperformed CMS and showed comparable performance to a tuned G1, while sustaining the highest throughput among tested collectors.
- Cassandra Performance: In read-intensive workloads, Platinum offered comparable performance to Shenandoah and significantly improved latency over CMS and tuned G1. For write-intensive workloads, improvements were still notable.
- Coupon Service Performance: Platinum demonstrated substantial improvements in tail latency for the coupon service, a critical component in Alibaba's platform.
Limitations and Future Work
- Major GC: Platinum primarily focuses on minor GC. The paper acknowledges that extending its design to handle major GC is a direction for future work.
- Spark Performance: Platinum's performance on Spark (a big-data processing framework) was slightly worse than G1. This is attributed to Spark's different memory and GC behavior, leading to more page faults and longer STW pauses, which Platinum's current design is not optimized for.
- Pinned Area Tuning: The size of the pinned area is a tunable parameter. While the paper suggests users can tune it based on application behavior, finding the optimal size might require experimentation.
- Hardware Dependency: The reliance on MPK means Platinum's full benefits are realized on hardware that supports this feature.
Summary updated