TL;DR
This paper tackles the challenge of designing 3D-stacked L2 caches for chip multiprocessors (CMPs) by exploring cache way placement strategies to balance performance and thermal management. They propose novel techniques like "shadow tags" for SRAM caches and "thermal-aware data migration" for MRAM caches, demonstrating a crucial trade-off between performance gains and thermal control efficiency.
What Problem Does This Paper Address?
As processors evolve into chip multiprocessors (CMPs) with increasing core counts and on-chip memory, the "memory wall" problem intensifies. Three-dimensional (3D) stacking offers a solution by allowing more memory to be integrated directly on top of logic, but this also exacerbates thermal issues due to higher power density. Specifically, this paper focuses on optimizing the design of 3D-stacked L2 caches, which are crucial for CMP performance. The challenge lies in how to organize and manage these caches across multiple layers to achieve both high performance and effective thermal control, considering different memory technologies like SRAM and MRAM.
Methods
The authors investigate various cache way placement strategies for 3D-stacked NUCA (Non-Uniform Cache Architecture) L2 caches. They explore:
- Baseline Placement: All ways of a cache index are placed in a single bank (traditional 2D approach).
- Distributed Placements:
- Uniform Distribution: Cache ways are spread evenly across all banks in a layer.
- Core-based Distribution: Cache ways are grouped into zones, with each zone corresponding to a core.
- Interlayer Core-based Distribution: Cache ways are distributed across layers, with each core having dedicated banks in different layers.
- Data Migration: Techniques to move data to cache lines closer to the accessing core to improve performance.
- Static Distributions: No data migration.
- Dynamic Distributions: Data migration is employed.
- Thermal Control Techniques:
- Selective-Working Method (for SRAM): Gating (disabling) subsets of cache banks during periods of low activity to reduce leakage power.
- Shadow Tags (for SRAM): A technique to predict miss rates when cache banks are gated, allowing for intelligent power gating without significant performance degradation.
- Thermal-Aware Data Migration (for MRAM): A policy to flatten power density distribution by alternating access between primary and secondary target banks to avoid hotspots, crucial for MRAM's dominant dynamic power consumption.
The research uses a 3D NUCA cache model integrated into the Simics simulator and employs the Hotspot 4.1 tool for thermal simulations.
Datasets
The paper uses a suite of benchmarks including those from SPEC OpenMP2001, NPB3.2, and SPEC2006 (referred to as "mix"). These benchmarks are run with one thread per core to simulate multiprogrammed workloads with varying transaction intensities to the L2 caches. No new benchmarks were introduced; instead, existing ones were used to evaluate the proposed designs.
Key Insights and Ideas
- 3D Stacking Opens New Placement Options: The vertical dimension in 3D stacking provides significantly more flexibility in cache way placement compared to traditional 2D designs, impacting both performance and thermal behavior.
- Placement Matters for Performance: Simply distributing cache ways doesn't guarantee performance gains. Static distributed placements can even degrade performance due to overheads and the LRU replacement policy not always keeping frequently accessed data close to the core.
- Data Migration is Key for Performance: Dynamic distributions with data migration significantly improve performance, especially for distributed cache designs, by ensuring data is closer to the accessing core.
- SRAM Thermal Control Focuses on Leakage: For SRAM, where leakage power is dominant, reducing power consumption through techniques like selective-working (power gating) is crucial for thermal management. Shadow tags offer an efficient way to implement this.
- MRAM Thermal Control Focuses on Dynamic Power: MRAM's dominant dynamic power consumption means thermal control must address power density distribution. Thermal-aware data migration is proposed to flatten this distribution and prevent hotspots.
- Performance vs. Thermal Trade-off is Inevitable: The study clearly demonstrates that optimizing for performance often comes at the cost of thermal efficiency, and vice-versa. The choice of design depends on the specific priorities.
- Interlayer Core-based Distribution is Promising: For performance, the interlayer core-based distribution with data migration appears to be the best option. For thermal management, it offers lower design complexity and less data loss.
Key Results
- Performance: Dynamic cache designs with data migration significantly outperform static designs. The interlayer core-based distribution with data migration achieved the best performance. Static distributed designs without migration showed degraded performance on some benchmarks.
- Thermal Control (SRAM): The selective-working method with shadow tags can reduce peak temperatures by 7-15 degrees Kelvin for benchmarks, with a performance degradation of less than 1%. Gating cache banks further away from the heat sink is more efficient.
- Thermal Control (MRAM): Variance in power density is significantly reduced with distributed placements, especially core-based distribution. Thermal-aware data migration, compared to original data migration, reduced peak temperatures by an average of 5.6 degrees Kelvin, with a performance degradation of less than 2%.
- Overhead: Shadow tags introduce a trivial area overhead (<3%) and power overhead (<5%). Thermal-aware data migration incurs a small performance overhead (one extra hop).
Limitations and Future Work
- Performance Degradation with Thermal Control: Both thermal control techniques (selective-working for SRAM and thermal-aware data migration for MRAM) lead to some performance degradation, highlighting the inherent trade-off.
- NoC Congestion Modeling: The paper notes that accurate congestion modeling in the NoC was not fully implemented due to infrastructure limitations, relying on approximations.
- MRAM Interlayer Placement: Interlayer placement was not explored for MRAM caches, as selective-working was deemed less critical due to low leakage power.
- Future Work: The authors suggest that the temperature reduction from selective-working could be more significant with an increased number of cache layers. They also imply that further exploration of thermal-aware data migration for MRAM and more sophisticated NoC modeling could yield further improvements. The research points towards the need for careful consideration of the performance-thermal trade-off when designing 3D-stacked caches.
Summary updated