TL;DR
Mako is a new transactional database system that achieves unprecedented throughput for geo-replicated, distributed transactions by decoupling transaction execution from replication. This allows Mako to speculatively execute transactions, masking the high latency of wide-area networks and outperforming existing systems by an order of magnitude.
What Problem Does This Paper Address?
Modern applications increasingly demand highly available, globally distributed data stores that can handle massive amounts of data and requests with strong consistency guarantees. Traditional distributed transactional systems, while ensuring consistency, suffer from significant performance bottlenecks due to the coordination overhead of distributed transactions and the latency of geo-replication. This coordination often becomes the limiting factor, making these systems orders of magnitude slower than single-machine transactional systems. Existing approaches to mitigate this, like faster networking, are largely ineffective for wide-area geo-replication. The core challenge is to achieve high throughput and strong consistency in geo-replicated distributed transactional systems without being crippled by network latency.
Methods
Mako's core innovation lies in decoupling transaction execution and replication. Instead of waiting for replication to complete before acknowledging a transaction, Mako executes transactions speculatively.
- Speculative Execution: Transactions are executed optimistically by shard leaders. Writes are buffered and become speculatively visible to subsequent transactions, even before they are replicated.
- Decoupled Replication: Replication happens in the background, independent of transaction execution. This allows Mako to continue processing new transactions without waiting for older ones to be replicated across datacenters.
- Speculative Two-Phase Commit (2PC): Mako uses 2PC for transaction coordination but speculatively. If a shard fails before replication completes, Mako employs a novel failure recovery mechanism to handle potential cascading aborts.
- Vector Clocks and Watermarks: Mako uses vector clocks to track dependencies between transactions and a decentralized vector watermark scheme to enable safe replay on followers and prevent unbounded cascading aborts during failures.
- Optimized for Geo-Replication: The design specifically targets the challenges of wide-area network latency, aiming to mask it through speculation.
Datasets
The paper does not mention specific datasets used for training. For evaluation, Mako is tested on the TPC-C benchmark and a microbenchmark simulating small-sized transactions. The TPC-C benchmark is configured to evaluate distributed transactions by varying the number of warehouses and cross-warehouse accesses. The microbenchmark focuses on read-only and read-modify-write transactions with varying ratios of cross-shard accesses.
Key Insights and Ideas
- Decoupling is Key: The most significant insight is that separating transaction execution from replication is crucial for achieving high throughput in geo-replicated systems. This allows speculation to effectively hide network latency.
- Speculation Masks Latency: By speculatively executing and making writes visible, Mako effectively masks the inherent latency of wide-area networks, a problem that traditional systems cannot overcome.
- Failure Recovery Without System Pauses: Mako's approach to failure recovery, which avoids pausing healthy shards, is a critical advancement over epoch-based commit protocols that can lead to system-wide stalls.
- Vector Clocks for Bounded Aborts: While Mako cannot prevent cascading aborts entirely, its use of vector clocks and watermarks effectively bounds their impact, preventing unbounded cascades.
- Trade-offs in Scalability: While Mako scales well, the paper highlights that vector clock compression, while necessary for extreme shard counts, can introduce performance degradations during failures.
Key Results
- Order of Magnitude Throughput Improvement: Mako achieves 8.6x higher throughput than state-of-the-art geo-replicated systems on the TPC-C benchmark with 10 shards.
- High Single-Shard Performance: Even on a single shard, Mako achieves 0.96M TPS, outperforming Calvin by 22.5x.
- Microbenchmark Performance: In a microbenchmark, Mako scales to 16.7M TPS with 10 shards, significantly outperforming OCC+OR.
- Latency: Mako's median latency on TPC-C with 10 shards and a batch size of 600 is 121 ms, with approximately 50 ms attributed to WAN RTT.
- Failure Recovery: Mako demonstrates efficient failure recovery, with healthy shards continuing operation with minimal disruption.
Limitations and Future Work
- No Read-Only Optimization on Followers: The current implementation does not allow read-only transactions to execute directly on followers, which is a common optimization. The authors suggest this could be added by maintaining special versions of keys.
- Vector Clock Compression Trade-offs: While Mako's vector clock implementation scales well, extreme compression (e.g., to a single timestamp) can degrade performance during failures. The default is a full-sized vector clock, with compression applied for very large numbers of shards.
- Shard Management: The current prototype uses static sharding. The authors acknowledge that dynamic sharding is an important area for future work, but argue that Mako's architecture simplifies this compared to prior systems.
- Event-Driven Threads: The use of event-driven threads for RPC processing, while efficient, can introduce additional latency.
Summary updated