Proposing a new task model towards many-core architecture

Akio Shimada, Balazs Gerofi, Atsushi Hori, Yutaka IshikawaMES 2013 · DOI

TL;DR

This paper introduces Partitioned Virtual Address Space (PVAS), a novel task model designed to overcome scalability issues in multi-task applications running on many-core processors. PVAS enhances inter-task communication and avoids virtual memory operation serialization, showing potential for significant performance improvements.

What Problem Does This Paper Address?

Modern computing is rapidly shifting towards many-core processors, which offer high throughput and power efficiency by packing numerous cores onto a single chip. To leverage these architectures, developers build multi-task applications using either multi-process or multi-thread models. However, these conventional models, originally designed for single-core systems, struggle to scale effectively on many-core processors.

The multi-process model isolates tasks in separate virtual address spaces. While this prevents interference, inter-process communication (IPC) requires data to be copied multiple times through intermediate buffers (often in the kernel), leading to high latency and performance bottlenecks, especially when many processes communicate frequently.

The multi-thread model allows tasks to share a single virtual address space, enabling direct data access and high-performance communication. However, this shared access creates race conditions when multiple threads concurrently modify shared data structures like memory region trees and page tables, forcing the operating system to serialize these critical virtual memory (VM) operations. This serialization, particularly for operations like mmap and munmap, becomes a significant performance bottleneck as the number of threads increases on many-core systems.

Methods

The paper proposes a new task model called Partitioned Virtual Address Space (PVAS). The core idea is to partition a single virtual address space into multiple regions, with each region assigned to a specific task (a "PVAS task").

Key aspects of the PVAS model include:

  • Partitioned Address Space: Each PVAS task gets its own dedicated partition within the overall virtual address space. This partition contains its TEXT, DATA&BSS, HEAP, and STACK segments.
  • Shared Access within PVAS: While tasks have their own partitions, they can still access data in other PVAS tasks' partitions directly, similar to the multi-thread model. This eliminates the need for intermediate buffers in inter-task communication.
  • Decoupled VM Management: To avoid the serialization issues of the multi-thread model, PVAS creates separate memory region trees for each PVAS partition. This means a PVAS task only updates the memory region tree for its own partition, preventing race conditions and serialization when multiple tasks perform mmap or munmap operations concurrently.
  • Page Table Tree Partitioning: The page table tree, which maps virtual to physical pages, is also logically partitioned into sub-trees corresponding to PVAS partitions. While the overall page table tree is shared, page fault handling operations are serialized only when multiple tasks attempt to update the same sub-tree. The authors argue this serialization is less frequent because tasks primarily access data within their own partitions.

Datasets

The paper does not discuss training datasets. For evaluation, they used:

  • Intel Xeon X5670 multi-core processor (6 cores × 2 Sockets): The hardware platform for their experiments.
  • Ping-pong communication benchmark: Implemented using MPI (specifically, a modified MPICH2 leveraging Nemesis) to measure inter-task communication latency.
  • Handmade VM operation benchmark: Designed to evaluate the performance of concurrent mmap, memset, and munmap operations.

Key Insights and Ideas

  • Bridging the Gap: PVAS offers a compelling hybrid approach, aiming to combine the direct data access benefits of multi-threading with the isolation benefits of multi-processing, while mitigating their respective drawbacks.
  • Decoupling VM Operations: The key innovation is the per-partition management of memory region trees, which directly addresses the serialization bottleneck in VM operations that plagues the multi-thread model.
  • Reduced Communication Overhead: By allowing direct access and eliminating intermediate buffers, PVAS significantly reduces the data copying required for inter-task communication, leading to lower latency.
  • Scalability Potential: The design inherently supports more concurrent VM operations and inter-task communications without serialization, suggesting better scalability on many-core systems.

Key Results

  • Inter-task Communication: The PVAS implementation showed better performance than the Nemesis rendezvous protocol in the ping-pong benchmark. This is attributed to PVAS requiring only one memory copy for communication, whereas the Nemesis rendezvous protocol requires two. For smaller message sizes, PVAS was faster than both eager and rendezvous protocols of Nemesis.
  • VM Operation Performance: The PVAS task model demonstrated performance comparable to the multi-process model and significantly better than the multi-thread model in the VM operation benchmark. This is because PVAS avoids the serialization of mmap and munmap operations, unlike the multi-thread model.

Limitations and Future Work

  • Micro Benchmarks: The authors acknowledge that preliminary evaluations were conducted using micro benchmarks (two tasks for ping-pong, twelve for VM operations) in a small environment.
  • Real-world Applications: The paper suggests that evaluating PVAS with real applications in large environments is crucial future work.
  • On-demand Paging in SMARTMAP: The paper briefly mentions that SMARTMAP avoids serialization by not supporting on-demand paging and using linear mapping, which incurs poor memory-use efficiency. This highlights a trade-off that PVAS might also need to consider in future optimizations.

Summary updated

BibTeX
@inproceedings{shimada2013proposing,
  author = {Akio Shimada and Balazs Gerofi and Atsushi Hori and Yutaka Ishikawa},
  title = {{Proposing a new task model towards many-core architecture}},
  booktitle = {MES},
  year = {2013},
  doi = {10.1145/2489068.2489075},
}
Figure