Dissecting Scale-Out Applications Performance on Diverse TLB Designs (S)

Tianmai Deng, Yixiao Xu, Chi Zhang, Zhengwei QiSEKE 2023 · DOI

Abstract

As cloud-native "scale-out" applications grow in complexity, the hidden cost of memory address translation is becoming a major performance bottleneck. This paper dissects how different Translation Lookaside Buffer (TLB) designs in modern Intel and AMD CPUs handle these workloads. Our findings reveal that while newer architectures have nearly eliminated instruction translation overhead, data translation remains a significant performance tax that can only be mitigated through high-associativity hardware and the aggressive use of huge pages.

Introduction

Modern scale-out applications—such as Spark for big data, Memcached for caching, or Nginx for web serving—are characterized by massive data sets and enormous instruction footprints. Every memory access requires the CPU to translate a virtual address into a physical one. This translation is cached in the TLB; a miss triggers a "page walk" through memory, which is significantly slower. While this overhead is known to account for 1.8% to 16% of runtime, the impact of specific hardware design choices—such as buffer size, associativity, and shared vs. split structures—on modern cloud workloads has remained under-explored.

Architectural Differences and Methodology

The researchers evaluated three distinct x86 architectures to compare design philosophies:

  • Intel 9th Gen (Coffee Lake): Represents older, more constrained TLB designs.
  • Intel 12th Gen (Golden Cove): Features increased TLB capacity and a split between "load" and "store" dTLBs.
  • AMD Zen3: Utilizes a fully associative L1 dTLB and a separate L2 iTLB/dTLB structure, contrasting with Intel’s merged L2 "sTLB" design.
Comparison of Intel and AMD MMU architectures showing merged vs. separate L2 TLB structures.

The study utilized the CloudSuite 4.0 benchmark suite, including workloads like Hadoop, Memcached, Cassandra, and Spark. Performance was measured using the Linux perf tool under two conditions: standard 4K memory pages and Transparent Huge Pages (THP), which uses 2MB chunks to reduce TLB entry requirements.

Performance Evaluation

The experimental results highlight a shift in where translation bottlenecks occur in modern systems.

The Shift from Code to Data

On the latest CPUs, such as Intel's 12th Gen, the instruction TLB (iTLB) has reached a capacity where code translation overhead has dropped below 0.1% for many applications. However, data translation (dTLB) remains a major bottleneck, with overheads reaching up to 10.9% of runtime.

iTLB and dTLB miss overhead percentages across various workloads, comparing baseline 4K pages and Transparent Huge Pages (THP) on Intel and AMD architectures.

Hardware Design Impact

  • Associativity: AMD’s Zen3 L1 dTLB, which is fully associative, outperformed Intel’s designs for "hot data" workloads, showing miss rates 36.7% to 47.4% lower than Intel counterparts.
  • Load/Store Splitting: Intel’s separation of "store" and "load" TLBs benefits load-heavy apps but penalizes "store-major" applications like Redis or Memcached due to the smaller size of the store-specific buffer.

The Role of Huge Pages

Enabling THP is critical for performance, reducing dTLB miss overhead by an average of 27.43% and iTLB misses by 12.16%. For data-intensive applications like Spark, the gains are even more pronounced.

Conclusion and Future Work

Scale-out applications continue to suffer an average data memory overhead of 3.68%, peaking at over 11%. While hardware improvements in newer generations (like the 39.6% reduction in code translation overhead from Intel 9th to 12th Gen) are promising, software-level optimizations like THP remain essential.

Future research is needed to address the "long-tail" latencies and memory bloat associated with THP. Furthermore, as most cloud applications run in virtualized environments, the "TLB tax" measured here represents a floor; nested paging in VMs likely exacerbates these overheads significantly.

Summary updated

BibTeX
@inproceedings{deng2023dissecting,
  author = {Tianmai Deng and Yixiao Xu and Chi Zhang and Zhengwei Qi},
  title = {{Dissecting Scale-Out Applications Performance on Diverse TLB Designs (S)}},
  booktitle = {SEKE},
  year = {2023},
  doi = {10.18293/seke2023-139},
}
Figure