TL;DR
This paper introduces a novel host-level I/O scheduler, hWA-BC, designed to eliminate performance interference between multiple users sharing Open-Channel SSDs. By analyzing I/O request sequentiality, hWA-BC intelligently identifies users contributing most to garbage collection (GC) overhead and penalizes them, thereby ensuring Quality of Service (QoS) for all users without requiring dedicated SSD streams.
What Problem Does This Paper Address?
In modern data centers and cloud environments, Solid State Drives (SSDs) are increasingly used due to their superior performance and energy efficiency compared to traditional Hard Disk Drives (HDDs). However, when multiple users or virtual machines (VMs) share a single SSD, performance interference can occur. This interference is exacerbated by internal SSD operations like garbage collection (GC), which consume I/O bandwidth and impact user-perceived performance. Existing solutions, such as the WA-BC scheduler, attempt to ensure Quality of Service (QoS) by allocating budgets to VMs. However, WA-BC has a significant drawback: it relies on misusing multi-stream SSDs to identify workload characteristics, which is inefficient and goes against the intended use of multi-stream SSDs. This paper addresses the need for a more efficient and effective host-level I/O scheduler that can provide performance isolation on Open-Channel SSDs without such limitations.
Methods
The core contribution of this paper is the hWA-BC (Host-Level Workload-Aware Budget Compensation) scheduler. Unlike previous approaches, hWA-BC operates at the host level and leverages the characteristics of Open-Channel SSDs, where the Flash Translation Layer (FTL) can be managed by the host.
Key methodological aspects include:
- Workload Characterization via I/O Sequentiality: Instead of relying on dedicated SSD streams (as WA-BC does), hWA-BC infers workload characteristics by observing the sequentiality of I/O requests. This is achieved by analyzing the continuity of Logical Page Numbers (LPNs) in write requests.
- GC Contribution Identification: The scheduler identifies which users contribute most to GC overhead. This is done by correlating workload sequentiality with GC activity. A higher degree of randomness in I/O requests (lower sequentiality) is assumed to lead to higher GC contributions.
- Cost Compensation/Penalization Algorithm: Based on the identified GC contribution, hWA-BC adjusts the cost of I/O operations. Users contributing more to GC face higher costs for their I/O requests, effectively penalizing them. Conversely, users with lower GC contributions receive a form of compensation, ensuring fairer resource allocation.
- GC Cost Separation: The scheduler explicitly separates the cost of GC operations from regular read/write operations. This allows for a more accurate assessment of the impact of GC on different workloads.
- Host-Level FTL Integration: By operating within the host's block I/O layer and utilizing the host-level FTL capabilities of Open-Channel SSDs (via the
pblkdriver), hWA-BC can reflect internal device states into its scheduling decisions. - Throttling Mechanism: When a container exhausts its budget, hWA-BC throttles its I/O requests by delaying their insertion into the write buffer (RWB), thus preventing further resource contention.
Datasets
The paper does not introduce new datasets. Instead, it uses standard benchmarking tools to simulate various workload patterns on an Open-Channel SSD.
- Benchmarking Tools:
fio(flexible I/O tester): Used for generating sequential and random read/write workloads.
Filebench: Used for simulating more realistic server workloads like Fileserver and Webproxy.- Hardware Platform:
- OpenSSD Cosmos+ board (used as the Open-Channel SSD).
- Host PC with Intel Core i7-4790K CPU and 8GB RAM running Ubuntu 16.04 LTS with Linux kernel v4.13.
Key Insights and Ideas
- I/O Sequentiality as a Proxy for GC Contribution: The paper demonstrates that the sequentiality of I/O requests is a surprisingly effective indicator of a user's contribution to SSD garbage collection overhead. This is a key insight that allows for workload characterization without complex per-user tracking of storage space.
- Open-Channel SSDs Enable Smarter Host-Level Scheduling: The architecture of Open-Channel SSDs, where the FTL is host-managed, is crucial. It allows host-level schedulers like hWA-BC to gain visibility into internal device operations and make more informed scheduling decisions.
- Targeted Penalization for Performance Isolation: By directly penalizing users who cause more GC, the scheduler can effectively mitigate performance interference. This is more nuanced than simply limiting bandwidth and directly addresses the root cause of interference in SSDs.
- Decoupling GC Cost: Explicitly separating GC costs from regular I/O costs allows for a fairer distribution of the overhead, especially benefiting read-heavy workloads that are less responsible for GC.
Key Results
The paper presents experimental results demonstrating the effectiveness of the hWA-BC scheduler in achieving performance isolation.
- Write Workload Patterns (Sequential vs. Random):
- In a scenario with sequential write and random write workloads, hWA-BC successfully provided higher throughput to the sequential write workload, which contributed less to GC. The scheduler allocated 10% of GC costs to sequential writes and 90% to random writes.
- Compared to
NoSched(no scheduling) andFairschedulers, hWA-BC showed a clearer separation in throughput between the two write patterns, indicating successful isolation. - Read-Write Workload Patterns:
- When running read and random write workloads concurrently, hWA-BC significantly improved the performance of the read workload compared to
NoSched.
- When running read and random write workloads concurrently, hWA-BC significantly improved the performance of the read workload compared to
- While the
Fairscheduler also improved read performance, it exhibited more throughput fluctuations and did not fully eliminate interference. hWA-BC, by removing GC costs from read operations, provided consistently higher throughput for the read workload. - Real Workload Experiments (Filebench + fio):
- In a mixed workload scenario with Fileserver, Webproxy, and fio (random write), hWA-BC provided the highest throughput to the Webproxy workload (read-heavy), followed by Fileserver, and then fio (write-only). This aligns with the scheduler's strategy of minimizing GC impact on read operations.
- The
NoSchedandFairschedulers showed more performance interference between the three workloads, with less distinct performance separation compared to hWA-BC.
Limitations and Future Work
- GC Cost Estimation: The paper acknowledges that the GC cost estimation is based on observations and may require further profiling for different SSDs and varying conditions. The formula for GC contribution is empirical.
- Workload Profiling Overhead: While the sequentiality-based approach is more efficient than WA-BC's method, there is still some overhead associated with tracking I/O request patterns.
- Future Work: The authors suggest that further research could explore more sophisticated methods for GC cost estimation and potentially integrate this host-level scheduler with device-level scheduling mechanisms for even greater performance gains. The paper also implies that adapting this approach to other types of storage devices or more complex multi-tenant environments could be a valuable direction.
Summary updated