VESPA: VIPT Enhancements for Superpage Accesses

Mayank Parasar, Abhishek Bhattacharjee, Tushar KrishnaarXiv 2017 · arXiv

Modern processor performance is heavily dependent on L1 cache efficiency. However, traditional cache designs face a structural bottleneck known as the Virtually-Indexed Physically-Tagged (VIPT) constraint. This paper, presented at ISCA 2017, introduces VESPA (VIPT Enhancements for Superpage Accesses), an architectural modification that leverages the increasing use of superpages to break this constraint, improving both speed and energy efficiency.

The VIPT Constraint

Most L1 caches use VIPT to overlap cache indexing with address translation in the Translation Lookaside Buffer (TLB). The limitation is that the index bits must come from the page offset. For standard 4KB pages, this is only 12 bits. If a cache line is 64 bytes, only 6 bits remain to select a set, restricting the cache to 64 sets. To grow the cache beyond 4KB, designers must increase associativity (adding more "ways" per set). High associativity, however, increases lookup latency and power consumption, as more hardware must be checked in parallel.

The VESPA Architecture

VESPA addresses this by exploiting superpages (e.g., 2MB or 1GB), which provide significantly more offset bits for indexing. The architecture organizes the L1 cache into multiple banks. For superpage accesses, VESPA uses the additional offset bits to index into a single bank, effectively reducing the active associativity and power for the common case.

Figure 1: Comparison between baseline VIPT and the VESPA architecture, showing how superpage offsets allow for smaller associative lookups.

When a memory request arrives, VESPA speculates that it is a superpage access. It uses the extra offset bits to index into just one bank. In parallel, the TLB confirms the page size. If it is a superpage hit, data is returned quickly from that single bank. If it is a standard 4KB page, the speculation fails, and the hardware triggers a lookup in the remaining banks during the next cycle.

Prevalence of Superpages

The effectiveness of VESPA relies on the frequency of superpage usage. The authors' characterization of modern operating systems (Linux, FreeBSD, Windows) shows that superpages are highly prevalent across various workloads, often accounting for 50% to over 90% of the memory footprint. This high prevalence ensures that the speculative "fast path" is the most frequent execution path.

Figure 6: Superpage memory footprint across various benchmarks, demonstrating the high prevalence of large pages in modern systems.

Key Results

Evaluation using the gem5 simulator and Cacti 6.5 power modeling shows significant gains:

  • Performance: Average Memory Access Time (AMAT) improves by 5-12% for 32-64KB caches and up to 19% for larger 128KB caches.
  • Energy Efficiency: Dynamic energy is reduced by 9-18%, and leakage energy is slashed by 95-98% due to faster execution and smaller active structures during lookups.
  • Transparency: The design requires modest hardware changes but zero changes to the operating system or application software stack.

Conclusion

VESPA successfully decouples L1 cache size from the associativity constraints of 4KB pages. By optimizing for the increasingly common superpage access path, it provides a scalable trajectory for larger, faster, and more energy-efficient L1 caches in future processors.

Summary updated

Abstract

L1 caches are critical to the performance of modern computer systems. Their design involves a delicate balance between fast lookups, high hit rates, low access energy, and simplicity of implementation. Unfortunately, constraints imposed by virtual memory make it difficult to satisfy all these attributes today. Specifically, the modern staple of supporting virtual-indexing and physical-tagging (VIPT) for parallel TLB-L1 lookups means that L1 caches are usually grown with greater associativity rather than sets. This compromises performance -- by degrading access times without significantly boosting hit rates -- and increases access energy. We propose VIPT Enhancements for SuperPage Accesses or VESPA in response. VESPA side-steps the traditional problems of VIPT by leveraging the increasing ubiquity of superpages; since superpages have more page offset bits, they can accommodate L1 cache organizations with more sets than baseline pages can. VESPA dynamically adapts to any OS distribution of page sizes to operate L1 caches with good access times, hit rates, and energy, for both single- and multi-threaded workloads. Since the hardware changes are modest, and there are no OS or application changes, VESPA is readily-implementable. By superpages (also called huge or large pages) we refer to any page sizes supported by the architecture bigger than baseline page size.
BibTeX
@misc{parasar2017vespa,
  author = {Mayank Parasar and Abhishek Bhattacharjee and Tushar Krishna},
  title = {{VESPA: VIPT Enhancements for Superpage Accesses}},
  year = {2017},
  eprint = {1701.03499},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1701.03499},
}
Figure