TL;DR
This paper tackles the challenge of ensuring atomic writes in database systems, crucial for data integrity. The authors propose a novel storage I/O subsystem that modifies the Linux EXT4 file system and CFQ I/O scheduler, coupled with a specially designed SSD, to guarantee atomic writes with significant performance improvements over traditional methods.
What Problem Does This Paper Address?
Database systems rely on ACID properties (Atomicity, Consistency, Isolation, Durability) to maintain data integrity. Atomicity, in particular, ensures that a transaction is treated as a single, indivisible unit – either all its operations complete successfully, or none of them do. Traditional database systems often use a "double write buffer" technique to achieve atomicity. This involves writing data twice: once to a dedicated buffer and then to its final location. While effective, this doubles the write I/O, impacting performance.
To overcome this, "atomic write" techniques have emerged, aiming to reduce write overhead. However, these techniques require specialized hardware (SSDs) and modified I/O subsystems (file systems and I/O schedulers) to guarantee atomicity at the storage level. Existing solutions, like FusionIO's, were proprietary and had limitations. This paper addresses the need for a more accessible and efficient atomic write solution by proposing modifications to widely used open-source components (EXT4, CFQ) and a conceptual design for an atomic write-supporting SSD.
Methods
The core of this paper lies in a multi-pronged approach to enable atomic writes:
- Atomic Write-Supporting SSD Design: The authors propose an SSD that can distinguish between normal and atomic write requests. This is achieved by using a flag in the spare area of flash memory pages. Crucially, the SSD is designed to ensure that all pages belonging to an atomic write request are written before their corresponding mapping table entries are updated. This prevents partial writes during power failures (SPOR - Sudden Power Off Recovery). A "count" field in the spare area also helps track the size of atomic write commands.
- Modified EXT4 File System: The EXT4 file system is modified to align block allocations with the database's write unit (16KB in MariaDB). This ensures that database data is allocated in contiguous blocks, preventing fragmentation that could lead to non-atomic writes. The
ext4_mb_normalize_requestfunction is altered to ensure allocated block sizes are multiples of 16KB. A new "atomic" flag is passed from the VFSdo_direct_IOfunction to signal atomic write requests. - Modified CFQ I/O Scheduler: The CFQ I/O scheduler is modified to prevent the merging of atomic write requests with normal write requests, or with requests from different database components (e.g., data vs. log files). This ensures that atomic write commands are treated as distinct, indivisible units. The
blk_rq_merge_okfunction is adjusted to allow merging of atomic write requests with other atomic write requests and normal write requests with other normal write requests, but not between the two types.
Datasets
The paper evaluates its proposed techniques using standard database benchmarking tools:
- TPC-c-mysql: A widely used benchmark for online transaction processing (OLTP) workloads, simulating a complex order-entry environment.
- SysBench: Another popular OLTP benchmark tool that can be configured to test database performance under various loads.
The authors used MariaDB (version 5.5.34) as the database system. They did not introduce new datasets but rather evaluated their system's performance on these established benchmarks.
Key Insights and Ideas
- The Power of Collaboration: The most significant insight is that achieving atomic writes efficiently requires a synergistic approach involving the database, the file system, the I/O scheduler, and the storage device (SSD). No single component can solve the problem alone.
- Leveraging Existing Infrastructure: Instead of creating entirely new systems, the paper demonstrates the effectiveness of modifying well-established open-source components like EXT4 and CFQ. This makes the proposed solution more practical and easier to adopt.
- SSD's Role in Atomicity: The design of the atomic write-supporting SSD is crucial. By intelligently using spare areas for flags and count information, and by ensuring atomic updates to mapping tables, the SSD can guarantee atomicity even in the face of power failures.
- Write Unit Alignment is Key: Aligning file system block allocations with the database's native write unit (16KB in this case) is vital for preventing fragmentation and ensuring that database writes can be treated as atomic units by the storage.
- Direct I/O Benefits: The use of direct I/O, bypassing the page cache, not only reduces write amplification but also simplifies the I/O path, contributing to performance gains.
Key Results
The experimental results demonstrate a significant performance improvement:
- Throughput Improvement: The proposed atomic write technique showed a performance improvement of 1.4 to 1.5 times compared to the traditional double write buffer technique.
- Reduced Write Amplification: The atomic write technique reduced the amount of data written to storage by approximately 40% to 50% compared to the double write buffer.
- TPC-c Benchmark: On TPC-c, the atomic write method achieved 1.2 to 1.5 times higher throughput than the double write buffer.
- SysBench Benchmark: SysBench results showed a 1.25x improvement for
no_doubleand a 1.6x improvement for the proposedatomicmethod over the double write buffer.
Limitations and Future Work
- SSD Dependency: The proposed atomic write solution requires a specially designed SSD that supports the atomic write interface. This is a hardware dependency that might limit immediate adoption.
- Database-Specific Tuning: While the paper focuses on MariaDB and its 16KB write unit, the effectiveness of the file system and scheduler modifications might vary for databases with different write unit sizes or I/O patterns.
- Fragmentation Management: The paper acknowledges that severe fragmentation in EXT4 could still pose a challenge for allocating contiguous blocks. They suggest using online defragmentation tools as a solution, but this adds an operational overhead.
- Future Work: The authors implicitly suggest that further research could explore optimizing the SSD design for even lower overhead and broader compatibility. Extending the file system and scheduler modifications to support other database systems and file systems would also be valuable.
Summary updated