Host-Level I/O Scheduler for Achieving Performance Isolation with Open-Channel SSDs

Sooyun Lee, Kyuhwa Han, Dongkun ShinJournal of KIISE 2020 · DOI

TL;DR

This paper introduces a novel host-level I/O scheduler, hWA-BC, designed to eliminate performance interference between multiple users sharing Open-Channel SSDs. By analyzing I/O request sequentiality, hWA-BC intelligently identifies users contributing most to garbage collection (GC) overhead and penalizes them, thereby ensuring Quality of Service (QoS) for all users without requiring dedicated SSD streams.

What Problem Does This Paper Address?

In modern data centers and cloud environments, Solid State Drives (SSDs) are increasingly used due to their superior performance and energy efficiency compared to traditional Hard Disk Drives (HDDs). However, when multiple users or virtual machines (VMs) share a single SSD, performance interference can occur. This interference is exacerbated by internal SSD operations like garbage collection (GC), which consume I/O bandwidth and impact user-perceived performance. Existing solutions, such as the WA-BC scheduler, attempt to ensure Quality of Service (QoS) by allocating budgets to VMs. However, WA-BC has a significant drawback: it relies on misusing multi-stream SSDs to identify workload characteristics, which is inefficient and goes against the intended use of multi-stream SSDs. This paper addresses the need for a more efficient and effective host-level I/O scheduler that can provide performance isolation on Open-Channel SSDs without such limitations.

Methods

The core contribution of this paper is the hWA-BC (Host-Level Workload-Aware Budget Compensation) scheduler. Unlike previous approaches, hWA-BC operates at the host level and leverages the characteristics of Open-Channel SSDs, where the Flash Translation Layer (FTL) can be managed by the host.

Key methodological aspects include:

  • Workload Characterization via I/O Sequentiality: Instead of relying on dedicated SSD streams (as WA-BC does), hWA-BC infers workload characteristics by observing the sequentiality of I/O requests. This is achieved by analyzing the continuity of Logical Page Numbers (LPNs) in write requests.
  • GC Contribution Identification: The scheduler identifies which users contribute most to GC overhead. This is done by correlating workload sequentiality with GC activity. A higher degree of randomness in I/O requests (lower sequentiality) is assumed to lead to higher GC contributions.
  • Cost Compensation/Penalization Algorithm: Based on the identified GC contribution, hWA-BC adjusts the cost of I/O operations. Users contributing more to GC face higher costs for their I/O requests, effectively penalizing them. Conversely, users with lower GC contributions receive a form of compensation, ensuring fairer resource allocation.
  • GC Cost Separation: The scheduler explicitly separates the cost of GC operations from regular read/write operations. This allows for a more accurate assessment of the impact of GC on different workloads.
  • Host-Level FTL Integration: By operating within the host's block I/O layer and utilizing the host-level FTL capabilities of Open-Channel SSDs (via the pblk driver), hWA-BC can reflect internal device states into its scheduling decisions.
  • Throttling Mechanism: When a container exhausts its budget, hWA-BC throttles its I/O requests by delaying their insertion into the write buffer (RWB), thus preventing further resource contention.

Datasets

The paper does not introduce new datasets. Instead, it uses standard benchmarking tools to simulate various workload patterns on an Open-Channel SSD.

  • Benchmarking Tools:
    • fio (flexible I/O tester): Used for generating sequential and random read/write workloads.
  • Filebench: Used for simulating more realistic server workloads like Fileserver and Webproxy.
  • Hardware Platform:
    • OpenSSD Cosmos+ board (used as the Open-Channel SSD).
  • Host PC with Intel Core i7-4790K CPU and 8GB RAM running Ubuntu 16.04 LTS with Linux kernel v4.13.

Key Insights and Ideas

  • I/O Sequentiality as a Proxy for GC Contribution: The paper demonstrates that the sequentiality of I/O requests is a surprisingly effective indicator of a user's contribution to SSD garbage collection overhead. This is a key insight that allows for workload characterization without complex per-user tracking of storage space.
  • Open-Channel SSDs Enable Smarter Host-Level Scheduling: The architecture of Open-Channel SSDs, where the FTL is host-managed, is crucial. It allows host-level schedulers like hWA-BC to gain visibility into internal device operations and make more informed scheduling decisions.
  • Targeted Penalization for Performance Isolation: By directly penalizing users who cause more GC, the scheduler can effectively mitigate performance interference. This is more nuanced than simply limiting bandwidth and directly addresses the root cause of interference in SSDs.
  • Decoupling GC Cost: Explicitly separating GC costs from regular I/O costs allows for a fairer distribution of the overhead, especially benefiting read-heavy workloads that are less responsible for GC.

Key Results

The paper presents experimental results demonstrating the effectiveness of the hWA-BC scheduler in achieving performance isolation.

  • Write Workload Patterns (Sequential vs. Random):
    • In a scenario with sequential write and random write workloads, hWA-BC successfully provided higher throughput to the sequential write workload, which contributed less to GC. The scheduler allocated 10% of GC costs to sequential writes and 90% to random writes.
  • Compared to NoSched (no scheduling) and Fair schedulers, hWA-BC showed a clearer separation in throughput between the two write patterns, indicating successful isolation.
  • Read-Write Workload Patterns:
    • When running read and random write workloads concurrently, hWA-BC significantly improved the performance of the read workload compared to NoSched.
  • While the Fair scheduler also improved read performance, it exhibited more throughput fluctuations and did not fully eliminate interference. hWA-BC, by removing GC costs from read operations, provided consistently higher throughput for the read workload.
  • Real Workload Experiments (Filebench + fio):
    • In a mixed workload scenario with Fileserver, Webproxy, and fio (random write), hWA-BC provided the highest throughput to the Webproxy workload (read-heavy), followed by Fileserver, and then fio (write-only). This aligns with the scheduler's strategy of minimizing GC impact on read operations.
  • The NoSched and Fair schedulers showed more performance interference between the three workloads, with less distinct performance separation compared to hWA-BC.

Limitations and Future Work

  • GC Cost Estimation: The paper acknowledges that the GC cost estimation is based on observations and may require further profiling for different SSDs and varying conditions. The formula for GC contribution is empirical.
  • Workload Profiling Overhead: While the sequentiality-based approach is more efficient than WA-BC's method, there is still some overhead associated with tracking I/O request patterns.
  • Future Work: The authors suggest that further research could explore more sophisticated methods for GC cost estimation and potentially integrate this host-level scheduler with device-level scheduling mechanisms for even greater performance gains. The paper also implies that adapting this approach to other types of storage devices or more complex multi-tenant environments could be a valuable direction.

Summary updated

Abstract

최근 솔리드 스테이트 드라이브(SSD)가 하드 디스크 드라이브(HDD)에 비해 높은 I/O 성능 및 낮은 에너지 소비를 제공함에 따라 여러 사용자가 자원을 공유하는 데이터센터 및 클라우드 컴퓨팅 환경에서 SSD의 사용이 확장되고 있다. 이러한 전환에 따라, 각 사용자에 서비스 품질(Quality of Service)을 보장하는 방법에 대한 연구가 활발히 이뤄지고 있다. 이전에 제안된 Workload-Aware Budget Compensation (WA-BC) 스케줄러는 NVMe SSD를 여러 가상머신이 공유하는 환경에서 각 가상머신의 QoS를 보장한다. 하지만 WA-BC 스케줄러는 가상머신별 워크로드 특징을 알아내기 위해 멀티 스트림 SSD를 부적절하게 사용한다는 점에서 취약하다. 우리는 이와 같은 취약점을 보완하여 오픈 채널 SSD를 여러 사용자가 공유하는 환경에서 사용자 간 성능 간섭을 제거하는 호스트 수준의 I/O 스케줄러를 제안한다. 제안된 스케줄러는 SSD 스트림을 사용자별로 할당하지 않고도 워크로드 특징을 I/O 요청의 연속성을 통해 파악한다. 이는 제안된 스케줄러가 호스트 수준의 스케줄러이나 오픈 채널 SSD의 특성을 활용하여 디바이스 내부의 정보를 스케줄링에 반영할 수 있기에 가능하다. 워크로드의 특성을 통해 SSD 내 I/O 간섭을 발생시키는 가비지 컬렉션에 대한 기여도를 파악하고, 기여도가 높은 사용자에 대해 처벌함으로써 우리는 스토리지 자원을 공유하는 여러 사용자 간에 발생하는 성능 간섭을 제거할 수 있음을 본 연구에서 보인다.
BibTeX
@article{lee2020host,
  author = {Sooyun Lee and Kyuhwa Han and Dongkun Shin},
  title = {{Host-Level I/O Scheduler for Achieving Performance Isolation with Open-Channel SSDs}},
  journal = {Journal of KIISE},
  year = {2020},
  doi = {10.5626/jok.2020.47.2.119},
}
Figure