Efficient Kernel Object Management for Tiered Memory Systems with KLOC

Sudarsun Kannan, Yujie Ren, Abhishek BhattacharjeearXiv 2020 · DOI · arXiv

Abstract

Software-controlled heterogeneous memory systems have the potential to improve performance, efficiency, and cost tradeoffs in emerging systems. Delivering on this promise requires efficient operating system (OS) mechanisms and policies for data management. Unfortunately, modern OSes do not support efficient tiering of data between heterogeneous memories. While this problem is known for application-level data pages, the question of how best to tier OS kernel objects has largely been ignored. We show that careful kernel object management is vital to the performance of software-controlled tiered memory systems. We find that state-of-the-art OS page management overlooks how best to tier, migrate, and manage kernel objects like inodes, dentry caches, and network buffers. In response, we characterize hotness, reuse, and liveness properties of kernel objects to develop KLOC (Kernel-Level Object Contexts). We evaluate KLOC using a real-system emulation framework on large-scale workloads like RocksDB, Redis, Cassandra, and Spark, achieving 1.4X to 4X higher throughput compared to prior art.

The Problem: The Forgotten Kernel Objects

Modern servers increasingly mix different types of memory: small amounts of very fast memory (like High Bandwidth Memory) and large amounts of slower, cheaper memory (like Persistent Memory). To optimize performance, the OS must decide which data belongs in the fast tier. Current solutions focus almost exclusively on application-level data, leaving a massive blind spot: kernel objects. These data structures manage filesystems and networking (e.g., buffer caches, journal blocks, socket buffers) and are critical to overall system performance.

Traditional page-management techniques fail for kernel objects for three primary reasons:

  1. Anonymity: Kernel objects are often shared across the system rather than owned by a single application.
  2. Short Lifespans: Many kernel objects live for only milliseconds, far shorter than the 30-second intervals typical of OS hotness scanners.
  3. High Overhead: Standard page-migration tools are too heavy for these tiny, transient objects.

Characterizing Kernel Object Dynamics

To design an effective tiering policy, we first characterized the memory footprint and temporal behavior of kernel objects across various workloads.

Kernel Objects are Heavy Hitters

In many I/O-intensive applications, kernel-level activity accounts for a significant portion of memory usage. As seen in the page allocation distribution, kernel buffers and caches represent a substantial fraction of the total memory footprint for applications like RocksDB and Cassandra.

Figure 1: Page allocation distribution across heap (app), page cache (OS), and kernel buffers.

The "Short Life" Revelation

Our analysis revealed that kernel objects are extremely short-lived. In workloads like RocksDB and Redis, cache pages often have lifetimes of less than 160ms, and kernel buffers expire in just 60ms. This proves that traditional, slow-moving hotness tracking is fundamentally mismatched with kernel object dynamics.

Figure 5: Object lifetime (in milliseconds) for kernel buffers and caches across different applications.

KLOC: Context-Aware Management

We propose KLOC (Kernel-Level Object Contexts) to address these challenges. Instead of managing memory page-by-page, KLOC groups kernel objects into logical "contexts" based on the entities they serve, such as a specific file, socket, or virtual device.

Key innovations include:

  • Semantic Hinting: KLOC uses system calls (like open(), read(), or socket()) as immediate signals to place associated objects in fast memory, bypassing slow background scanners.
  • Lightweight Object Mapping: A red-black tree-based map tracks object-context associations, allowing the OS to quickly find and migrate "cold" objects when fast memory is needed.
  • Migration Support for Slab Pages: By using vmalloc() to create virtually contiguous regions, KLOC enables the migration of small kernel objects that the standard Linux slab allocator cannot move.

Key Results

KLOC achieved 1.4x to 4x higher throughput compared to Nimble, the previous state-of-the-art for tiered memory management. Specific improvements included a 1.58x gain for RocksDB and a 3.3x improvement for the network-heavy Redis. By proactively moving inactive kernel metadata to the slow tier, KLOC frees up critical fast memory for application data, significantly improving overall system efficiency.

Conclusion

As heterogeneous memory becomes standard in data centers, ignoring the kernel's memory footprint is no longer an option. KLOC demonstrates that by treating kernel objects as first-class citizens in the memory hierarchy and managing them via logical contexts, we can unlock significant performance gains that traditional page-based tiering misses.

Summary updated

Abstract

Software-controlled heterogeneous memory systems have the potential to improve performance, efficiency, and cost tradeoffs in emerging systems. Delivering on this promise requires an efficient operating system (OS) mechanisms and policies for data management. Unfortunately, modern OSes do not support efficient tiering of data between heterogeneous memories. While this problem is known (and is being studied) for application-level data pages, the question of how best to tier OS kernel objects has largely been ignored. We show that careful kernel object management is vital to the performance of software-controlled tiered memory systems. We find that the state-of-the-art OS page management research leaves considerable performance on the table by overlooking how best to tier, migrate, and manage kernel objects like inodes, dentry caches, journal blocks, network socket buffers, etc., associated with the filesystem and networking stack. In response, we characterize hotness, reuse, and liveness properties of kernel objects to develop appropriate tiering/migration mechanisms and policies. We evaluate our proposal using a real-system emulation framework on large-scale workloads like RocksDB, Redis, Cassandra, and Spark and achieve 1.4X to 4X higher throughput compared to the prior art.
BibTeX
@misc{kannan2020efficient,
  author = {Sudarsun Kannan and Yujie Ren and Abhishek Bhattacharjee},
  title = {{Efficient Kernel Object Management for Tiered Memory Systems with KLOC}},
  year = {2020},
  eprint = {2004.04760},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.04760},
}

Loading related content...

Figure