TL;DR
Blizzard is a novel block storage system designed to bring cloud-scale performance and availability to traditional, cloud-oblivious applications. It achieves this by using a unique nested striping scheme and a fast-flush mechanism that allows applications to achieve high throughput without sacrificing crash consistency.
What Problem Does This Paper Address?
Migrating traditional applications (like databases or email servers) to the cloud often involves a trade-off: either run them in virtual machines with limited storage performance (50-250 MB/s) or rewrite them to use cloud-native storage APIs, which is complex and limits customization. Existing cloud storage solutions, while scalable for big-data workloads, don't offer the fine-grained consistency and high IOPS that traditional applications expect. This gap prevents many enterprises from fully leveraging cloud infrastructure for their existing software. Blizzard aims to bridge this gap by providing a high-performance, cloud-scale block storage interface that "just works" with unmodified POSIX/Win32 applications.
Methods
Blizzard employs several key techniques:
- Nested Striping: Instead of simply striping data across disks, Blizzard stripes data across "segments," where each segment is composed of multiple "tracts" (FDS's unit of storage). This nested approach allows for aggressive sharding and better disk parallelism, especially for sequential workloads, by distributing data across many disks and avoiding convoy dilation.
- Fast Flushes with Prefix Write Commits: To overcome the performance bottleneck of traditional
fsync()operations (which act as write barriers), Blizzard acknowledges flush requests immediately. It then asynchronously writes data to remote disks, respecting the ordering constraints of flush epochs. This "eventual durability" provides high write performance while still guaranteeing crash consistency. - Log-based Out-of-Order Commits: For maximum write throughput, Blizzard can commit writes out-of-order. It uses a log structure and a deterministic permutation to ensure that even if writes are not durable in the order they were issued, the system can recover to a consistent state by replaying the log.
- Full-Bisection Bandwidth Network: Blizzard relies on a network with no oversubscription, ensuring that any client can communicate with any server at full NIC speed, which is crucial for its aggressive striping and performance goals.
Datasets
Not discussed in the paper. The paper focuses on microbenchmarks and synthetic workloads to evaluate performance and reliability.
Key Insights and Ideas
- Cloud-scale performance for legacy apps is achievable: The paper demonstrates that it's possible to deliver significant performance improvements (2x-10x) to unmodified, traditional applications without requiring them to be rewritten for the cloud.
- Decoupling durability from ordering is key: By separating the acknowledgment of a flush from its actual durability, Blizzard unlocks significant write performance gains, a concept borrowed from optimistic crash consistency.
- Nested striping is crucial for parallelism: The novel nested striping mechanism effectively combats convoy dilation and maximizes disk parallelism, especially for sequential I/O, which is a common bottleneck in distributed storage.
- Network topology matters immensely: The reliance on a full-bisection bandwidth network is highlighted as a critical enabler for Blizzard's performance, underscoring the importance of network infrastructure in achieving cloud-scale storage.
Key Results
- Throughput: Blizzard can achieve sequential write throughputs of 700 MB/s and sequential read/random read/random write throughputs exceeding 1000 MB/s, significantly outperforming traditional local disks and cloud offerings like EBS.
- Latency: Blizzard's read latencies were 2x-4x lower than EBS, and write latencies with fast acknowledgments were 7x-14x faster than EBS.
- Application Performance: Macrobenchmarks showed Blizzard improving unmodified application performance by 2x-10x, with notable gains for I/O-intensive tasks like DirCopy and Grep.
- Reliability: In all tested crash scenarios, Blizzard consistently recovered to a prefix-consistent state of the virtual disk.
- Data Loss: While fast acknowledgment schemes introduce a risk of data loss for the most recent writes, the log-based scheme offers better control and scalability, especially under high IOPS.
Limitations and Future Work
- Network Sensitivity: Blizzard's performance is highly dependent on a low-latency, full-bisection bandwidth network. Adding even a few milliseconds of latency significantly degrades performance.
- Read-before-write penalty: The log-based commit scheme incurs a read-before-write penalty for writes smaller than the block size, which can impact performance if clients frequently issue misaligned writes.
- Future Work: The paper mentions that role-based striping can be used to control disk parallelism for different users in private cloud deployments. Further exploration of this and other optimizations for specific workloads could be future directions. The authors also note that their current implementation does not discuss disconnected operation further due to space constraints, suggesting this could be an area for future investigation.
Summary updated