Boosting File Systems Elegantly: A Transparent NVM Write-ahead Log for Disk File Systems

Guoyu Wang, Xilong Che, Haoyang Wei, Shuo Chen, Puyi He, Jiawei Hu

TL;DR

NVLog is a lightweight, transparent write-ahead log that uses Non-Volatile Memory (NVM) to accelerate traditional disk file systems (like Ext4 and XFS) by up to 15x. By intelligently separating synchronous writes into NVM while keeping asynchronous traffic in the fast DRAM page cache, it provides massive performance gains without requiring data migration or complex architectural changes.

What Problem Does This Paper Address?

Modern storage systems face a "Goldilocks" problem. Traditional disks (SSDs/HDDs) are slow, especially for synchronous writes (where an app waits for data to be "safe" on disk). Non-Volatile Memory (NVM) is incredibly fast and persistent, but it has much lower capacity than disks and is slower than DRAM.

Existing solutions to bridge this gap usually fall into two traps:

  1. NVM-Specific File Systems (e.g., NOVA): These require moving all your data to NVM. Because NVM is smaller than disk, this isn't scalable. Furthermore, they often bypass the DRAM page cache, which can actually make them slower than traditional systems for certain workloads.
  2. Overlay Systems (e.g., SPFS): These sit on top of disk file systems but often introduce heavy indexing overhead or accidentally slow down "normal" asynchronous writes by forcing them through NVM.

The authors of NVLog sought a "transparent" middle ground: how can we use a tiny bit of NVM to make existing, time-tested file systems feel like they are running at NVM speeds?

Methods

NVLog introduces a "log-aside" architecture. Instead of replacing the file system, it plugs into the Linux Virtual File System (VFS) layer.

  • On-Demand Sync Absorption: NVLog only intercepts synchronous writes (like fsync or O_SYNC). Asynchronous writes continue to use the standard, highly optimized DRAM-to-disk path.
  • Efficient Log Structure: It uses a simple append-only log on the NVM. It differentiates between In-Place (IP) entries for small, byte-level writes (avoiding write amplification) and Out-of-Place (OOP) entries for large, page-aligned writes.
  • Active Sync Optimization: A heuristic algorithm predicts whether a file's access pattern would benefit from byte-level NVM logging or page-level disk writes, dynamically toggling flags to minimize overhead.
  • Heterogeneous Crash Consistency: In a system where data lives in DRAM, NVM, and Disk, a crash can lead to "version mismatch." NVLog records "write-back events" in its NVM log to track when data has safely reached the disk, ensuring that during recovery, the system doesn't accidentally roll back to an older version of a file.
  • Background Garbage Collection: A kernel thread periodically reclaims NVM space once the logged data has been successfully persisted to the underlying disk.

Datasets

The researchers evaluated NVLog using a variety of industry-standard benchmarks:

  • Microbenchmarks: FIO for raw throughput and latency across different read/write ratios.
  • Macrobenchmarks:
    • Filebench: Simulating fileserver, webserver, and mail server (Varmail) workloads.
  • RocksDB: Testing a high-performance Key-Value store (LSM-tree based).
  • SQLite: Using the YCSB benchmark to simulate real-world database transactions.
  • Hardware: Tests were performed on a server with Intel Xeon processors, 128GB of DRAM, and 256GB of Intel Optane Persistent Memory (NVM).

Key Insights and Ideas

  • DRAM is still King: The authors realized that NVM-native file systems often perform worse than legacy systems because they ignore the efficiency of the DRAM page cache. NVLog’s "aha moment" was deciding not to use NVM for everything, but only for the "slow path" (sync writes).
  • Recording > Retrieval: Most NVM systems waste time building complex indexes so they can read data back from NVM quickly. NVLog assumes the DRAM cache will handle most reads, so it optimizes the NVM strictly for fast recording (logging), deferring index building to the recovery phase.
  • Transparency is a Feature: By implementing NVLog within the VFS and MM (Memory Management) subsystems, it works with Ext4, XFS, and F2FS without changing a single line of their code.

Key Results

  • Massive Acceleration: NVLog accelerated Ext4 by up to 15.09x and XFS by 13.54x in pure synchronous write tests.
  • Beating the "Natives": In mixed workloads, NVLog outperformed NOVA (a dedicated NVM file system) by up to 3.72x because NVLog better utilized the DRAM page cache.
  • Efficiency: It outperformed SPFS (a leading overlay system) by a staggering 324x in certain random access scenarios, primarily because it avoided SPFS's heavy indexing overhead.
  • Small Footprint: Thanks to aggressive garbage collection, NVLog typically occupies less than 1% of the total write volume in NVM space.
  • Fast Recovery: Even after a crash, NVLog typically recovers the file system in about 10 seconds.

Limitations and Future Work

  • Cold I/O: NVLog does not accelerate "cold" reads (data not already in the DRAM cache), as it still has to fetch that data from the slower disk.
  • Hardware Saturation: The performance gains are capped by the write bandwidth of the NVM hardware itself. In systems with very few NVM modules, the NVM can become a bottleneck under extreme multi-threaded pressure.
  • Large Writes: For very large synchronous writes (e.g., 16KB+), the software stack overhead of the underlying file system begins to diminish the relative benefits of NVLog.
  • Future Direction: The authors suggest exploring how NVLog could be adapted for emerging NVM technologies like RRAM or MRAM, which have even smaller capacities but higher endurance.

Summary updated

Abstract

We propose NVLog, an NVM-based write-ahead log for disk file systems, designed to transparently harness the high performance of NVM within the legacy storage stack. NVLog provides on-demand byte-granularity sync absorption, reserving the fast DRAM path for asynchronous operations, meanwhile occupying NVM space only temporarily. To accomplish this, we designed a highly efficient log structure, developed mechanisms to address heterogeneous crash consistency, optimized for small writes, and implemented robust crash recovery and garbage collection methods. Compared to previous solutions, NVLog is lighter, more stable, and delivers higher performance, all while leveraging the mature kernel software stack and avoiding data migration overhead. Experimental results demonstrate that NVLog can accelerate disk file systems by up to 15.09x and outperform NOVA and SPFS in various scenarios by up to 3.72x and 324.11x, respectively.
BibTeX
@misc{wang2024boosting,
  author = {Guoyu Wang and Xilong Che and Haoyang Wei and Shuo Chen and Puyi He and Jiawei Hu},
  title = {{Boosting File Systems Elegantly: A Transparent NVM Write-ahead Log for Disk File Systems}},
  year = {2024},
  eprint = {2408.02911},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.02911},
}
Figure