Modern processor performance is heavily dependent on L1 cache efficiency. However, traditional cache designs face a structural bottleneck known as the Virtually-Indexed Physically-Tagged (VIPT) constraint. This paper, presented at ISCA 2017, introduces VESPA (VIPT Enhancements for Superpage Accesses), an architectural modification that leverages the increasing use of superpages to break this constraint, improving both speed and energy efficiency.
The VIPT Constraint
Most L1 caches use VIPT to overlap cache indexing with address translation in the Translation Lookaside Buffer (TLB). The limitation is that the index bits must come from the page offset. For standard 4KB pages, this is only 12 bits. If a cache line is 64 bytes, only 6 bits remain to select a set, restricting the cache to 64 sets. To grow the cache beyond 4KB, designers must increase associativity (adding more "ways" per set). High associativity, however, increases lookup latency and power consumption, as more hardware must be checked in parallel.
The VESPA Architecture
VESPA addresses this by exploiting superpages (e.g., 2MB or 1GB), which provide significantly more offset bits for indexing. The architecture organizes the L1 cache into multiple banks. For superpage accesses, VESPA uses the additional offset bits to index into a single bank, effectively reducing the active associativity and power for the common case.
When a memory request arrives, VESPA speculates that it is a superpage access. It uses the extra offset bits to index into just one bank. In parallel, the TLB confirms the page size. If it is a superpage hit, data is returned quickly from that single bank. If it is a standard 4KB page, the speculation fails, and the hardware triggers a lookup in the remaining banks during the next cycle.
Prevalence of Superpages
The effectiveness of VESPA relies on the frequency of superpage usage. The authors' characterization of modern operating systems (Linux, FreeBSD, Windows) shows that superpages are highly prevalent across various workloads, often accounting for 50% to over 90% of the memory footprint. This high prevalence ensures that the speculative "fast path" is the most frequent execution path.
Key Results
Evaluation using the gem5 simulator and Cacti 6.5 power modeling shows significant gains:
- Performance: Average Memory Access Time (AMAT) improves by 5-12% for 32-64KB caches and up to 19% for larger 128KB caches.
- Energy Efficiency: Dynamic energy is reduced by 9-18%, and leakage energy is slashed by 95-98% due to faster execution and smaller active structures during lookups.
- Transparency: The design requires modest hardware changes but zero changes to the operating system or application software stack.
Conclusion
VESPA successfully decouples L1 cache size from the associativity constraints of 4KB pages. By optimizing for the increasingly common superpage access path, it provides a scalable trajectory for larger, faster, and more energy-efficient L1 caches in future processors.
Summary updated