Exploiting Semantics of Virtual Memory to Improve the Efficiency of the On-Chip Memory System

Bin Li, Zhen Fang, Li Zhao, Xiaowei Jiang, Lin Li, Ruoying He, Ravishankar Iyer, Srihari Makineni, Ravi Iyer 0001Lecture Notes in Computer Science 2012 · DOI · OpenAlex

Abstract

Modern memory management units (MMUs) typically treat all virtual memory addresses as semantically equivalent once translated to physical addresses. This "semantic blindness" leads to inefficiencies in Translation Lookaside Buffer (TLB) management and on-chip interconnect usage. This paper proposes two hardware-software co-design optimizations that exploit the unique properties of stack memory—which is thread-private and highly accessed. By introducing "Anticipatory Superpaging" and "Stack-Aware Cache Placement," the authors demonstrate a reduction in TLB misses by up to 20% and a decrease in interconnect power consumption by over 14%, achieving these gains with minimal hardware overhead.

Introduction: The Semantic Gap in Memory Management

Modern computer systems distinguish between different memory regions at the software level, such as the stack (private to a thread) and the heap (shared dynamic data). However, once these addresses are translated, the hardware loses this context.

Typical virtual memory layout showing the stack and heap regions.

This lack of semantic awareness causes two primary issues:

  1. TLB Thrashing: Frequent stack accesses can be evicted from the TLB by less critical heap entries, leading to performance penalties.
  2. Interconnect Waste: In multi-core processors, data is often interleaved across the chip to balance load. For private stack data, moving it to a remote cache slice increases latency and wastes power.

Proposed Methods

The authors introduce two mechanisms requiring minimal changes to the operating system and hardware:

1. Anticipatory Superpaging

Traditional superpaging involves expensive data copying to promote regular pages. Instead, the kernel identifies stack regions during the initial page fault and proactively allocates a superpage. This increases "TLB reach" immediately, allowing a single TLB entry to cover more memory without traditional overhead.

2. Stack-Aware Cache Placement

A "stack hint" bit is added to the page table and TLB. When an L1 cache miss occurs, the hardware checks this bit to determine routing.

Comparison of baseline cacheline interleaving vs. the proposed stack-aware function.

As shown in the figure above, if the "Stack hint" is present, the request is routed directly to the local Last-Level Cache (LLC) slice. Non-stack references continue to use standard interleaving. This keeps private data physically close to the requesting core, reducing ring hops and power consumption.

Evaluation and Results

The designs were evaluated using the SPEC CPU2000 benchmark suite on ManySim, a cycle-accurate simulator modeling a 16-core Chip Multiprocessor (CMP). Power consumption was estimated using the Orion 2.0 model.

Key Findings

  • Access Density: While the stack represents only ~21% of the memory footprint, it accounts for ~40% of all memory references, making it an ideal candidate for optimization.
  • TLB Efficiency: Anticipatory Superpaging reduced data TLB misses by 10% to 20% on average for low-power configurations.
  • Power Savings: Stack-Aware Cache Placement reduced "ring hops" by 20%, leading to a 20.6% saving in total ring power in multi-program environments.
  • Performance: The system achieved a 1.7% improvement in total instruction throughput, demonstrating that power efficiency does not compromise speed.

Discussion and Limitations

The authors note that while benefits are clear, some benchmarks (e.g., applu) showed increased TLB misses if stack entries evicted heap entries prematurely. Future work includes extending this semantic-aware approach to other regions like thread-local storage or read-only code segments.

Summary updated

BibTeX
@inproceedings{li2012exploiting,
  author = {Bin Li and Zhen Fang and Li Zhao and Xiaowei Jiang and Lin Li and Ruoying He and Ravishankar Iyer and Srihari Makineni and Ravi Iyer},
  title = {{Exploiting Semantics of Virtual Memory to Improve the Efficiency of the On-Chip Memory System}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2012},
  doi = {10.1007/978-3-642-32820-6\_24},
}
Figure