Reducing L1 caches power by exploiting software semantics

Zhen Fang, Zhao Li, Xiaowei Jiang, Shih‐Lien Lu, Ravi Iyer, Tong Li, Seung Eun Lee2012 · DOI

TL;DR

This paper introduces a novel approach to reduce power consumption in L1 caches by exploiting software semantics. By tagging cache lines with information about their virtual memory region (stack/heap) and privilege level (user/kernel), the cache can intelligently skip unnecessary searches, leading to significant power savings without impacting performance.

What Problem Does This Paper Address?

High-performance processors rely heavily on L1 caches for speed, but their current design is power-inefficient. L1 caches are typically set-associative, meaning that when a memory request arrives, all "ways" (parallel storage locations) within a selected set are searched simultaneously. This is done using physical addresses. However, this approach ignores crucial software semantics:

  • Virtual Memory Bifurcation: The virtual address space often distinguishes between stack and heap memory. A stack access will never hit data in a heap region, and vice-versa.
  • Privilege Ring Levels: User-mode instructions will never reside in kernel-mode code segments, and vice-versa.

Because the cache doesn't understand these distinctions, it wastes energy by searching ways that are guaranteed not to contain the requested data. This is particularly problematic for L1 caches where performance is paramount, and aggressive power-saving techniques used in L2/L3 caches (like serialized access) are not feasible.

Methods

The authors propose two main techniques to exploit software semantics for power reduction:

  1. Vtag (Virtual Memory Tag) for Data Caches (DL1):
    • Each cache line in the DL1 is augmented with a single bit representing the most significant bit (MSB) of its virtual address. This bit effectively distinguishes between stack and heap regions.
  2. When a data read request arrives, this Vtag is checked in parallel with the standard TLB (Translation Lookaside Buffer) access and physical tag lookup.
  3. If the Vtag of the incoming request does not match the Vtag of a cache line in a particular way, that way's data subbank is not accessed. This eliminates bitline discharges and sense amplifier operations for those ways.
  4. This design is implemented to allow selective data readout, avoiding unnecessary bitline activity.
  5. Rtag (Ring Level Tag) for Instruction Caches (IL1):
    • Each cache line in the IL1 is tagged with the privilege ring level (user or kernel) at which it was fetched.
  6. When an instruction fetch request arrives, the Rtag is used to determine which ways could contain the requested instruction. For example, if the request is from user mode, only ways tagged as user-mode code are searched.
  7. To avoid impacting the critical path, an optimized approach uses a "bitmask" per set. This bitmask is generated during cache line replacement or when the processor's ring level changes, and it directly controls way selection for subsequent accesses. This decouples the Rtag comparison from the critical read path.

A key advantage of both Vtag and Rtag is that they are checked concurrently with existing operations (TLB access for Vtag, and during cache line installation for the optimized Rtag), thus avoiding any increase in cache hit latency.

Datasets

The evaluation uses traces from:

  • SPEC CPU2000: A suite of benchmarks for evaluating computer system performance.
  • SPECjappsServer04: A Java-based server application benchmark suite.

These traces were collected using a full-system emulator (SoftSDV) and captured dynamic instruction sequences for both user and kernel code.

Key Insights and Ideas

  • Software Semantics are Underutilized: Despite their clear implications for memory access patterns, software semantics like virtual memory regions and privilege levels are largely ignored by traditional L1 cache designs.
  • Kernel Code Occupies Significant IL1 Space: Counter-intuitively, kernel code can occupy a substantial portion of the L1 instruction cache, even during compute-intensive user applications. This is due to the nature of OS operations (e.g., decision trees) versus user code (e.g., tight loops).
  • Targeted Cache Access Saves Power: By intelligently filtering cache accesses based on software semantics, a significant number of unnecessary tag comparisons and data array accesses can be avoided, directly reducing dynamic power consumption.
  • Performance Neutrality is Achievable: The proposed Vtag and Rtag mechanisms are designed to operate in parallel with existing cache operations, ensuring no increase in hit latency.

Key Results

  • DL1 Power Savings: Vtag reduces cache block fetches from DL1 by an average of 27% for SPEC CPU benchmarks. This translates to an average power saving of 15% for DL1.
  • IL1 Power Savings: Rtag is even more effective, reducing IL1 cache block fetches by an average of 57%. This results in an average power saving of over 30% for IL1.
  • Overhead: The storage overhead for Vtag and Rtag is minimal (64 bytes for DL1, 128 bytes for IL1 on a Nehalem-like processor), and the added logic is small.
  • Performance: The proposed designs do not impact performance and are transparent to application software.

Limitations and Future Work

  • Address Aliasing: For DL1's Vtag, address aliasing (where multiple virtual pages map to the same physical page) can lead to false misses. The authors suggest that the OS can disable Vtag usage for such mappings, and that unintentional aliasing is rare.
  • Write Operations: Vtag does not optimize write operations, which are counted as 100% matches in selectivity statistics.
  • Future Work: The paper suggests that the principles of semantic-aware caching can be extended to other cache levels (L2/L3) and other architectural components. Further exploration of more complex software semantics could yield additional power savings.

Summary updated

Abstract

To access a set-associative L1 cache in a high-performance processor, all ways of the selected set are searched and fetched in parallel using physical address bits. Such a cache is oblivious of memory references' software semantics such as stack-heap bifurcation of the memory space, and user-kernel ring levels. This constitutes a waste of energy since e.g., a user-mode instruction fetch will never hit a cache block that contains kernel code. Similarly, a stack access will not hit a cacheline that contains heap data.
BibTeX
@misc{fang2012reducing,
  author = {Zhen Fang and Zhao Li and Xiaowei Jiang and Shih‐Lien Lu and Ravi Iyer and Tong Li and Seung Eun Lee},
  title = {{Reducing L1 caches power by exploiting software semantics}},
  year = {2012},
  doi = {10.1145/2333660.2333750},
}

Loading related content...

Figure