LPNS: Scalable and Latency-Predictable Local Storage Virtualization for Unpredictable NVMe SSDs in Clouds

Bo Peng, Cheng Guo, Jianguo Yao, Haibing GuanUSENIX ATC 2023 · OpenReview

TL;DR

This paper introduces LPNS, a novel storage virtualization system designed to provide predictable low latency for NVMe SSDs in cloud environments. By addressing the interference issues between virtual machines sharing the same SSD, LPNS ensures Quality of Service (QoS) for latency-sensitive applications without significantly sacrificing overall throughput.

What Problem Does This Paper Address?

Modern cloud platforms increasingly rely on NVMe SSDs for their high performance. However, a fundamental challenge arises when multiple tenants (virtual machines, containers) share these SSDs. While existing virtualization techniques focus on maximizing throughput, they often fail to guarantee predictable latency for individual tenants. This "device-level interference" can cause latency-sensitive workloads to miss their Service Level Agreements (SLAs), even when the underlying SSDs are capable of much lower latencies. The paper aims to bridge this gap by providing a solution that offers both scalability and latency predictability for NVMe storage in the cloud.

Methods

LPNS is an OS-level NVMe virtualization system built upon a mediated pass-through mechanism. Its core innovations include:

  • Self-Feedback Control: LPNS continuously monitors I/O performance and workload characteristics of virtual machines. This feedback loop allows it to adaptively adjust resource allocation and scheduling to maintain predictable latency.
  • Flexible I/O Queue and Command Scheduling: LPNS introduces dynamic queue management, differentiating between exclusive queues for latency-sensitive VMs and shared queues for others. It also implements fine-grained command throttling to prevent high-throughput workloads from overwhelming latency-sensitive ones.
  • Deterministic Network Calculus: The authors leverage deterministic network calculus to mathematically model and derive theoretical worst-case latency bounds for virtualized storage. This provides a formal guarantee for the predictable latency.
  • Scalable Polling Design: LPNS employs an efficient polling mechanism to reduce I/O latency by avoiding interrupt overhead. It can adaptively trigger polling threads based on workload activity, balancing performance and CPU utilization.
  • Hardware Queue Pools: LPNS organizes hardware I/O queues into pools, allowing for flexible mapping and multiplexing between physical queues and virtual machines, enhancing scalability.

Datasets

The paper does not explicitly mention or introduce new datasets. Instead, it uses standard benchmarks and real-world traces for evaluation:

  • Micro-benchmarks: FIO (Flexible I/O Tester) is used to generate various random read/write workloads with different parameters (e.g., numjobs, iodepth).
  • Application Benchmarks:
    • YCSB (Yahoo! Cloud Serving Benchmark) with RocksDB is used to simulate key-value store workloads.
  • Real-world webuser server traces from Florida International University (FIU) are replayed.

Key Insights and Ideas

  • Device-level interference is a critical bottleneck: The paper convincingly demonstrates that even with advanced virtualization, the contention for NVMe SSD resources among multiple tenants is a primary cause of unpredictable latency.
  • Predictable latency doesn't have to kill throughput: LPNS shows that by intelligently managing I/O queues and commands, it's possible to achieve significant latency improvements with only a modest reduction in overall throughput for competing workloads.
  • Formal modeling is key to guarantees: The use of deterministic network calculus provides a strong theoretical foundation for the latency bounds, moving beyond empirical observations.
  • Adaptive control is essential for dynamic clouds: The self-feedback mechanism allows LPNS to adapt to changing workload patterns, a crucial feature for modern cloud environments.

Key Results

  • Latency Optimization: LPNS achieves significant latency optimization, with reported improvements of up to 18.72x over state-of-the-art NVMe virtualization techniques like SR-IOV and SPDK. In micro-benchmarks, it can bound latency to under 25 µs on P5800X SSDs and under 90 µs on PM1735 SSDs.
  • Throughput: LPNS can increase up to 1.45x additional total throughput over K2 with max_inflight=32 (equivalent to 47.66% of the maximum throughput of the P5800X SSD). The throughput loss for competing workloads is generally kept below 7.07%.
  • Scalability: LPNS demonstrates good scalability, supporting multiple virtual machines sharing an SSD while maintaining predictable latency for sensitive workloads.
  • Real-world Performance: In real-world webuser workload tests, LPNS closely tracks the baseline latency (when a single VM monopolizes the SSD), successfully bounding latency within 50 µs on P5800X.

Limitations and Future Work

  • Parameter Tuning: The deterministic network calculus model requires tuning for different NVMe SSDs or block sizes. The authors note that for SSDs with significantly lower random write IOPS than random read, LPNS might sacrifice more throughput.
  • Trade-off Management: Achieving a perfect balance between latency predictability and throughput sacrifice is challenging and depends on the specific SSD and workload. The parameter Ω needs careful selection by administrators.
  • Future Work: The authors plan to improve the balance between latency prediction and throughput on more types of NVMe SSDs. They also aim to integrate LPNS into a hardware/software co-designed solution to further offload overhead and enhance scheduling efficiency.

Summary updated

BibTeX
@inproceedings{peng2023lpns,
  author = {Bo Peng and Cheng Guo and Jianguo Yao and Haibing Guan},
  title = {{LPNS: Scalable and Latency-Predictable Local Storage Virtualization for Unpredictable NVMe SSDs in Clouds}},
  booktitle = {USENIX ATC},
  year = {2023},
  url = {https://openreview.net/forum?id=X5mEcv9L5E},
}
Figure