Revisiting virtual memory for high performance computing on manycore architectures: a hybrid segmentation kernel approach

Yuki Soma, Balazs Gerofi, Yutaka IshikawaROSS@ICS 2014 · DOI

TL;DR

This paper proposes a novel hybrid kernel architecture for many-core CPUs that combines segmentation and paging memory management. By offloading applications to cores running a lightweight segmentation kernel while keeping a full paging kernel on dedicated cores, they achieve significant performance improvements for memory-intensive HPC workloads by eliminating costly TLB misses.

What Problem Does This Paper Address?

Modern operating systems heavily rely on page-based virtual memory management (paging) for features like memory fragmentation avoidance and fine-grained access control. However, paging involves translating virtual to physical addresses using page tables, which reside in main memory. This translation process, even with Translation Lookaside Buffers (TLBs) to cache mappings, incurs overhead. This overhead becomes particularly problematic for High-Performance Computing (HPC) applications that often deal with massive datasets and exhibit complex, non-local memory access patterns. These patterns lead to frequent TLB misses, significantly impacting performance and power consumption. Furthermore, on many-core architectures, maintaining TLB consistency across numerous cores (TLB shoot-downs) adds another layer of overhead. The paper aims to mitigate these paging-related performance bottlenecks for HPC workloads on many-core systems.

Methods

The core contribution is a hybrid segmentation kernel approach. Instead of running all applications on a single, monolithic paging kernel, this design partitions CPU cores. A dedicated set of cores runs a full-featured paging kernel (referred to as the "paging kernel"), providing the rich OS services. The majority of the cores are dedicated to running HPC applications under a lightweight "segmentation kernel." This segmentation kernel provides a minimalistic execution environment that leverages segmentation-based memory management, which avoids the overheads associated with paging (TLB misses, page walks).

When an application running on a segmentation core needs an OS service (e.g., a system call), the segmentation kernel delegates this request to the paging kernel. This delegation mechanism allows applications to benefit from the performance advantages of segmentation while still having access to the full functionality of paging-based OS services when needed. The paper details the design of this hybrid kernel, including how memory is mapped and how system calls are delegated between the two kernel types.

Datasets

Not discussed in the paper. The evaluation uses standard benchmarks.

Key Insights and Ideas

  • Segmentation for HPC is Viable: The paper demonstrates that segmentation, often overlooked in modern OS design, can offer substantial performance benefits for memory-intensive HPC workloads by eliminating TLB-related overheads.
  • Hybrid Approach is Key: The success lies in the hybrid nature. Applications get the speed of segmentation, while the OS retains the flexibility and robustness of paging for its own operations and for applications that might still require it.
  • Core Specialization is Powerful: On many-core architectures, dedicating cores to specific tasks (paging vs. segmentation) is an effective strategy to optimize performance, a luxury not afforded by traditional single- or multi-core systems.
  • TLB Misses are a Major Bottleneck: The results strongly suggest that for certain workloads, the cost of TLB misses and page walks can dominate execution time, making segmentation a compelling alternative.
  • Future of Segmentation: The paper advocates for renewed consideration of full segmentation support in 64-bit architectures, given its potential for HPC.

Key Results

The paper presents compelling performance improvements for HPC applications:

  • Random Memory Access Benchmark: Segmentation showed significant speedups over both 4KB and 2MB pages. At 2GB memory usage, segmentation was up to 22% faster than 4KB pages and 26% faster than 2MB pages. The performance gap widened considerably with increasing memory footprint.
  • Graph500 Benchmark: For large graph scales (SCALE 25), the hybrid segmentation approach yielded up to 81% improvement compared to 4KB pages and 9% improvement compared to 2MB pages on Intel's MPSS Linux.

Limitations and Future Work

  • 32-bit Mode Restriction: The current implementation is restricted to 32-bit mode because full segmentation support without paging is not available in 64-bit mode on current x86 architectures. This limits application memory usage to under 4GB.
  • Single-Threaded Applications: The prototype currently supports only single-threaded applications.
  • Linker/Compiler Modifications: While the design aims for a similar interface, some applications might require linker script modifications to set the text section start address to 0.
  • Future Work: The authors plan to:
  • Provide support for multi-threading in the segmentation kernel.
  • Further evaluate applications not only for performance but also for OS noise.
  • Advocate for hardware vendors to consider full segmentation support in 64-bit execution modes.

Summary updated

BibTeX
@inproceedings{soma2014revisiting,
  author = {Yuki Soma and Balazs Gerofi and Yutaka Ishikawa},
  title = {{Revisiting virtual memory for high performance computing on manycore architectures: a hybrid segmentation kernel approach}},
  booktitle = {ROSS@ICS},
  year = {2014},
  doi = {10.1145/2612262.2612264},
}
Figure