TL;DR
This paper argues that current server architectures are inefficient for memcached, a critical component of modern internet services. They propose "Thin Servers with Smart Pipes" (TSSP), an SoC design that offloads GET requests to a dedicated hardware accelerator, achieving a 6-16x improvement in power-performance compared to conventional servers.
What Problem Does This Paper Address?
Modern internet services rely heavily on distributed in-memory key-value stores like memcached for scalability. However, current deployments use expensive, high-end processors that are surprisingly inefficient for memcached workloads. This inefficiency stems not from raw compute or memory bandwidth limitations, but from the overhead of processing network packets within the OS kernel and NIC. This leads to wasted power and limits the overall capacity of data centers, which are often constrained by power delivery. The paper aims to identify these bottlenecks and propose a more efficient architecture.
Methods
The authors conducted a detailed architectural characterization of memcached performance and power consumption. They used a custom load-testing framework to simulate realistic traffic patterns, varying object sizes and access frequencies. They then analyzed performance counters and microarchitectural metrics on both high-performance (Xeon) and low-power (Atom) server configurations with different Network Interface Cards (NICs). Based on these findings, they propose the Thin Servers with Smart Pipes (TSSP) architecture, which integrates a low-power CPU with a dedicated hardware accelerator for memcached GET requests within a System-on-Chip (SoC). This accelerator handles network packet processing and data lookups, offloading these tasks from the CPU. They evaluated the TSSP concept using an FPGA prototype.
Datasets
The paper does not explicitly list or use pre-existing datasets for training. Instead, they created a suite of five carefully designed memcached workloads to emulate real-world traffic. These workloads vary in object size distribution (e.g., FixedSize, MicroBlog, Wiki, ImgPreview, FriendFeed) and access patterns (e.g., single GET vs. MULTI-GET requests), aiming to expose a broad range of memcached behavior and performance characteristics. They state that these workloads exhibit similar statistical properties to those reported for production clusters at Facebook.
Key Insights and Ideas
- Network Stack is the Bottleneck: The primary performance limitation for memcached is not CPU processing power or memory bandwidth, but the overhead of the OS kernel and NIC in processing each network packet.
- Inefficient CPU Utilization: Even high-end CPUs are significantly underutilized, operating at a fraction of their theoretical instruction throughput due to frequent trips into the networking stack, leading to poor instruction cache and branch predictor performance.
- Low-Power Cores Aren't a Silver Bullet: While low-power cores offer energy savings, their current implementations are too inefficient at handling the network-intensive memcached workload to provide a satisfactory power-performance trade-off.
- NIC Quality Matters More Than Bandwidth: Advanced NIC features like segment offload and multi-queue support significantly improve performance, often more than simply increasing raw bandwidth.
- Hardware Acceleration for GET Requests: Offloading the most frequent operation (GET requests) to a dedicated hardware accelerator within an SoC can drastically improve both performance and power efficiency.
Key Results
- Significant Inefficiency: Xeon processors operate at less than 1/8th and Atom processors at less than 1/16th of their theoretical instruction throughput for memcached.
- Instruction Cache Thrashing: Memcached exhibits an instruction cache miss rate up to 15x worse than typical benchmarks, primarily due to the large code footprint of the Linux kernel and networking stack.
- NIC Feature Impact: Enterprise-class NICs with features like software batching and multi-queue support provide 70-110% performance improvements over commodity NICs.
- TSSP Performance: The TSSP architecture, demonstrated on an FPGA prototype, achieves a 6-16x improvement in power-performance compared to conventional Xeon and Atom-based server baselines. For the FixedSize workload, TSSP achieved 282 KOps/s at 16W, while Xeon and Atom systems consumed 143W and 35W respectively, with lower throughput.
- UDP vs. TCP: While using UDP instead of TCP can improve performance for some workloads, the primary gains in TSSP come from hardware acceleration, not just protocol choice.
Limitations and Future Work
- FPGA Prototype Limitations: The TSSP evaluation was based on an FPGA prototype, which required some compromises, such as limiting key sizes and value sizes, and implementing memory management in software. The authors note that a custom ASIC implementation would likely yield further efficiency gains.
- Software-Hardware Split: While TSSP offloads GET requests to hardware, other operations like SET, memory allocation, and eviction are handled by software. The authors acknowledge that implementing these complex operations in hardware would be challenging.
- Future Work: The authors suggest that their TSSP design could be applied to other data- and network-centric workloads. They also imply that further optimization of the hardware accelerator and tighter integration within the SoC could lead to even greater improvements. The paper also hints at the potential for custom ASICs for maximum efficiency.
Summary updated