TL;DR
Persistent Memory (PM) offers a significant performance boost for storage, but existing file mapping techniques, designed for slower storage, are a major bottleneck. This paper introduces and evaluates new PM-optimized file mapping strategies, demonstrating that a novel hash-based approach, HashFS, significantly outperforms traditional methods, leading to substantial throughput gains in real-world applications.
What Problem Does This Paper Address?
Persistent Memory (PM) is a game-changer for storage, bridging the gap between slow traditional storage and fast volatile RAM. Its byte-addressable nature and near-DRAM latency make it ideal for high-performance file systems. However, while many file system components have been re-architected for PM, the critical "file mapping" process—translating logical file offsets to physical storage locations—has largely been overlooked. This process can consume up to 70% of the I/O path in PM-optimized file systems, yet existing methods often reuse decade-old designs or lack rigorous analysis. This paper aims to fill this gap by exploring the design space of PM-optimized file mapping and proposing a superior solution.
Methods
The authors explore a range of file mapping approaches, categorizing them into per-file and global strategies.
- Per-File Mapping: They adapt traditional extent trees and radix trees for PM. These structures maintain mapping information on a per-file basis. The key PM optimizations include operating directly on PM (bypassing the page cache) and using cursors in DRAM for locality.
- Global Mapping: Recognizing the limitations of per-file approaches, they propose two global hash-table-based strategies.
- Global Cuckoo Hash Table: This approach uses two hash functions per entry for fast lookups and a mechanism to resolve collisions by evicting and relocating entries. It also includes a fixed-size translation cache in DRAM for sequential access acceleration.
- HashFS: This is their proposed novel approach. It's a global hash table that also integrates block allocation. It uses linear probing for collision resolution and leverages SIMD parallelism for vector hash operations. A key design choice is that the hash table index directly corresponds to the physical block location, simplifying the process.
Datasets
The paper does not introduce new datasets. Instead, it uses:
- Microbenchmarks: To isolate and evaluate specific aspects of file mapping performance (e.g., latency, fragmentation impact, file size impact, IO size impact, space utilization, concurrency).
- Application Workloads:
- Filebench: Used to simulate real-world file system workloads (fileserver for write-heavy, webproxy for read-heavy) to assess performance under more realistic conditions.
- YCSB (Yahoo! Cloud Serving Benchmark) on LevelDB: Used to evaluate end-to-end performance in key-value workloads, a common benchmark for PM-optimized file systems.
Key Insights and Ideas
- File Mapping is a Major Bottleneck: The paper strongly emphasizes that file mapping, not just block allocation or data movement, is a significant performance limiter in PM file systems, especially for smaller, random I/O operations.
- Page Caching is Detrimental for PM File Mapping: Contrary to traditional file system wisdom, the authors demonstrate that using a page cache in DRAM for PM file mapping structures offers no benefit and can even be detrimental due to overhead. Direct access to PM is superior.
- Global Mapping Outperforms Per-File: Global hash table approaches generally show better performance, especially in scenarios with low locality (cold caches), as they avoid the multi-level traversals inherent in tree-based structures.
- HashFS is the Star Performer: The proposed HashFS, with its integrated block allocation and linear probing, consistently delivers the best performance across various benchmarks, particularly excelling in insertion operations and offering substantial throughput gains.
- Fragmentation is a Problem for Trees, Not Hashes: Extent trees, in particular, suffer significantly from file system fragmentation, while hash-based approaches are largely unaffected.
- File System Specificity Matters: Using general-purpose PM storage structures (like level hashing) directly for file mapping is less effective than PM-optimized, file-system-specific designs like HashFS.
Key Results
- Throughput Gains: HashFS achieved up to a 45% increase in throughput over the state-of-the-art Strata PM-optimized file system (using page-cached extent trees) on YCSB workloads.
- Latency Reductions: In cold cache scenarios, global file mapping was up to 15x faster than per-file mapping. HashFS was 14-20x faster than other structures in this scenario.
- Fragmentation Resilience: HashFS was unaffected by fragmentation, while extent trees showed significant performance degradation.
- File Size Scalability: HashFS and other global hash tables showed static mapping overhead regardless of file size, unlike per-file tree structures which scaled with file size.
- IO Size Impact: As IO size increased, the performance gap between tree structures and hash tables narrowed, with HashFS consistently showing the lowest insertion times.
- Space Utilization: Even at 95% file system utilization, global hash tables still outperformed per-file mapping structures.
Limitations and Future Work
- Limitations:
- The authors acknowledge that while HashFS is generally superior, it can degrade in performance relative to global cuckoo hashing at very high space utilization (95%), though it still outperforms per-file methods.
- The study focuses on specific PM technologies (Intel Optane DC PM modules).
- Future Work:
- The authors suggest that their findings are generalizable to other PM file systems and that their proposed HashFS could be further optimized with different batching mechanisms or PM file systems that don't batch updates.
- They also briefly touch upon the potential for using existing PM storage structures but conclude that file-system-specific optimizations are crucial.
Summary updated