Lessons learned from the early performance evaluation of Intel Optane DC Persistent Memory in DBMS

Yinjun Wu, Kwanghyun Park, Rathijit Sen, Brian Kroth, Jaeyoung Do

TL;DR

This paper dives deep into the performance characteristics of Intel Optane DC Persistent Memory (PMem) when integrated into database management systems (DBMS). The authors reveal that simply swapping PMem for DRAM or using it as a direct replacement for storage isn't enough; specific DBMS design changes are crucial for unlocking its potential, particularly concerning I/O parallelism and data placement.

What Problem Does This Paper Address?

The advent of Non-Volatile Memory (NVM), exemplified by Intel's Optane DC Persistent Memory (PMem), presents a tantalizing opportunity for database systems. PMem offers the persistence of traditional storage (like SSDs) combined with the low latency and byte-addressability of DRAM. This unique blend could revolutionize DBMS design by allowing for larger working sets, faster persistent storage, or a combination of both. However, effectively leveraging PMem requires a fundamental understanding of its performance nuances, which differ significantly from both DRAM and SSDs. Before developers undertake costly redesigns, a clear picture of PMem's behavior under various configurations and workloads is essential. This paper aims to provide that clarity by evaluating PMem's performance in micro-benchmarks and real-world database workloads, offering practical insights for DBMS architects.

Methods

The authors employ a multi-pronged approach:

  • Micro-benchmarking: They use tools like Flexible I/O tester (fio) to meticulously measure PMem's performance across different configurations (DAX mode, File I/O mode, and Memory Mode) and varying parameters such as I/O request size, number of threads, and read/write operations. This allows for a granular understanding of the device's raw capabilities and limitations.
  • Database Workload Evaluation: They integrate PMem into Microsoft SQL Server 2019 and evaluate its performance using standard industry benchmarks: TPC-C (for Online Transaction Processing - OLTP) and TPC-H (for Online Analytical Processing - OLAP). This step bridges the gap between raw device performance and its impact on actual database operations.
  • Configuration Analysis: The paper systematically explores three primary PMem configurations:
  • Memory Mode: PMem acts as main memory, with DRAM serving as an L4 cache. This mode allows for increased memory capacity without application modification but sacrifices PMem's persistence.
  • DAX Mode: PMem is accessed directly via memory semantics (load/store instructions) after OS setup, offering low-latency persistent storage. This requires application-level awareness of potential issues like torn writes.
  • File I/O Mode: PMem is accessed through standard file system APIs, similar to traditional block devices, but with PMem's underlying characteristics.

Datasets

The paper does not introduce new datasets. Instead, it utilizes:

  • Micro-benchmark synthetic data: Generated by the Flexible I/O tester (fio).
  • Standard database benchmarks: TPC-C and TPC-H, with varying scale factors (SF) to simulate different data sizes (e.g., SF=100, SF=500 for TPC-H; SF=1300, SF=6500 for TPC-C). These scale factors generate database instances of approximately 100GB and 500GB.

Key Insights and Ideas

  • Small I/O Sizes are King for PMem DAX: PMem in DAX mode shines with very small I/O sizes (<4KB), offering significantly higher throughput than File I/O mode and SSDs. This suggests opportunities for DBMSs to optimize by using smaller page sizes.
  • Concurrency Scaling is Different: Unlike SSDs, PMem doesn't scale well with a large number of concurrent I/O threads. Performance degrades after a certain point, indicating a need to limit parallelism in DBMS designs.
  • Read/Write Asymmetry is Real: PMem exhibits a significant performance asymmetry between reads and writes, with reads being much faster. This is a critical factor for write-intensive workloads.
  • Memory Mode is Suboptimal for DBMS: Using PMem in Memory Mode as a direct replacement for DRAM in a DBMS buffer pool is problematic. While it increases capacity, the lack of persistence and the performance asymmetry (especially with writes to temporary storage) lead to worse performance than traditional DRAM buffer pools.
  • Simple Swaps Don't Cut It: Simply replacing DRAM with PMem in Memory Mode or using PMem as persistent storage without intelligent design changes (like optimized page placement policies) yields suboptimal results.

Key Results

  • Micro-benchmarks:
    • PMem DAX achieved up to 8x and 50x higher throughput than PMem File I/O and SSD, respectively, for 512B I/O sizes in sequential reads.
  • PMem DAX and File I/O showed a performance asymmetry, with peak read bandwidth around 10.6GB/s versus peak write bandwidth around 3.5GB/s, whereas SSDs were more balanced (~2.7GB/s read vs. ~2.8GB/s write).
  • PMem performance degraded after 12 threads for reads and 4 threads for writes in sequential access.
  • Database Workloads (Microsoft SQL Server 2019):
    • TPC-H (OLAP): With smaller datasets (SF=100), PMem DAX was significantly slower than SSD and PMem File I/O for Query 3 (6x slower). This was attributed to PMem DAX's inability to effectively cache hot pages in DRAM compared to File I/O and SSD.
  • TPC-H (OLAP): With larger datasets (SF=500), PMem DAX was slightly better than SSD.
  • TPC-C (OLTP): No significant performance difference was observed between PMem DAX, PMem File I/O, and SSD. This was explained by the write-intensive nature of TPC-C, where PMem's write performance limitations became a bottleneck.
  • SSD + PMem Memory Mode: This configuration performed worse than SSD + DRAM for TPC-H, especially with larger datasets, due to write operations to temporary storage.

Limitations and Future Work

  • Limitations:
    • The study focuses on a specific PMem device (Intel Optane DC Persistent Memory) and a particular DBMS (Microsoft SQL Server 2019). Results might vary with different hardware or software.
  • Due to space constraints, not all micro-benchmark results (e.g., latency) and workload evaluations are presented in full detail.
  • The write performance measurement for a 500GB dataset in PMem Memory Mode was not possible due to system crashes.
  • Future Work:
    • The authors emphasize the need for further research into PMem-aware page placement policies for DBMS buffer pools, especially for DAX mode.
  • They suggest revisiting earlier research on limiting write operations in PMem-aware DBMS designs (e.g., B-tree modifications, join algorithms) to assess their feasibility on real PMem hardware.
  • There are many open questions regarding the optimal use of PMem for other DBMS internals like access methods, logging, and recovery.

Summary updated

Abstract

Non-volatile memory (NVM) is an emerging technology, which has the persistence characteristics of large capacity storage devices(e.g., HDDs and SSDs), while providing the low access latency and byte-addressablity of traditional DRAM memory. This unique combination of features open up several new design considerations when building database management systems (DBMSs), such as replacing DRAM (as the main working space memory) or block devices (as the persistent storage), or complementing both at the same time for several DBMS components (such as access methods,storage engine, buffer management, logging/recovery, etc). However, interacting with NVM requires changes to application software to best use the device (e.g. mmap and clflush of small cache lines instead of write and fsync of large page buffers). Before introducing (potentially major) code changes to the DBMS for NVM, developers need a clear understanding of NVM performance in various conditions to help make better design choices. In this paper, we provide extensive performance evaluations conducted with a recently released NVM device, Intel Optane DC Persistent Memory (PMem), under different configurations with several micro-benchmark tools. Further, we evaluate OLTP and OLAP database workloads (i.e., TPC-C and TPC-H) with Microsoft SQL Server 2019 when using the NVM device as an in-memory buffer pool or persistent storage. From the lessons learned we share some recommendations for future DBMS design with PMem, e.g.simple hardware or software changes are not enough for the best use of PMem in DBMSs.
BibTeX
@inproceedings{wu2020lessons,
  author = {Yinjun Wu and Kwanghyun Park and Rathijit Sen and Brian Kroth and Jaeyoung Do},
  title = {{Lessons learned from the early performance evaluation of Intel Optane DC Persistent Memory in DBMS}},
  booktitle = {DaMoN},
  year = {2020},
  doi = {10.1145/3399666.3399898},
}
Figure