Dynamic voltage and frequency scaling for shared resources in multicore processor designs

Xi Chen, Zheng Xu, Hyungjun Kim, Paul V. Gratz, Jiang Hu, Michael Kishinevsky, Ümit Y. Ogras, Raid Ayoub2013 · DOI

TL;DR

This paper tackles the challenge of efficiently managing power for shared resources like on-chip networks (NOCs) and last-level caches (LLCs) in increasingly complex multicore processors. They introduce novel dynamic voltage and frequency scaling (DVFS) techniques that treat these shared resources as a single power domain, achieving significant energy savings (56% reduction in energy-delay product) with minimal performance impact.

What Problem Does This Paper Address?

Modern multicore processors are packed with more cores, leading to a growing proportion of the chip's area and power consumption being dedicated to shared resources like the on-chip communication fabric (Network-on-Chip or NOC) and the Last-Level Cache (LLC). These shared components are critical for overall chip performance, but their power demands can be substantial, especially when workloads are light or bursty. Traditional DVFS approaches often treat individual cores as separate power domains, which can lead to high overhead when managing shared resources and may not be optimal since these resources are accessed collectively. The challenge is to dynamically scale the voltage and frequency of these shared NOC and LLC resources to match their actual demand, thereby saving power without significantly degrading overall system performance.

Methods

The authors propose three novel DVFS techniques for the shared NOC and LLC resources, treating them as a single voltage/frequency (V/F) domain.

  • Throughput-Driven DVFS: This method uses the uncore's output throughput (data processed) as a dynamic reference point, aiming to match it with the input data injection rate from the cores. To handle situations where throughput might be artificially low (e.g., due to stalled cores), they introduce a mechanism to estimate the "effective" injection rate by considering stalled cores. This approach aims to keep the uncore busy but not over-provisioned.
  • Latency-Based DVFS (with Model Assist): This technique focuses on a "critical latency" metric, which aims to capture the uncore's impact on overall chip performance. It combines uncore latency with a criticality factor (based on load instructions and L1 cache misses) to better reflect performance bottlenecks. A model-assisted approach is used to predict future uncore behavior and dynamically adjust the target latency reference point, making it more adaptive to changing program phases.
  • Latency-Based PI Control with Dynamic Reference (OurBest): This is presented as their most advanced method. It builds upon the critical latency metric but employs a PI controller with a truly dynamic reference point. This reference is derived from a theoretical model of uncore latency, accounting for factors like propagation, serialization, and LLC access, while also incorporating a small buffer for network congestion. This approach aims for a more precise and adaptive control of the uncore V/F levels.

All methods utilize a Proportional-Integral (PI) controller, a standard control mechanism known for its simplicity and stability.

Datasets

The paper evaluates its proposed techniques using the PARSEC benchmark suite. These are standard, publicly available benchmarks designed for shared-memory multicore systems. The authors do not introduce new benchmarks but rather use these to simulate realistic workloads and assess the performance and power impact of their DVFS strategies.

Key Insights and Ideas

  • Shared Resources as a Single V/F Domain: Treating the entire NOC and LLC as a single power domain simplifies control and avoids inter-domain overhead, which is a significant advantage over core-centric approaches.
  • Dynamic Reference Points are Crucial: Fixed reference points for DVFS are insufficient for multicore systems with varying workloads. The paper demonstrates the need for dynamic reference points that adapt to program phases and system behavior.
  • Critical Latency is a Better Metric: A metric that combines uncore latency with its criticality to overall system performance (like their "critical latency") is more effective for power management than purely throughput-based or raw latency metrics.
  • Predictive Control Improves Efficiency: Model-assisted control, which attempts to predict future system behavior, can further enhance energy savings by proactively adjusting V/F levels.
  • System Behavior is Highly Dynamic: The supplementary material highlights that even within a single application, system behavior (loads, misses) can change drastically between control intervals, underscoring the challenge of accurate prediction and the need for robust, adaptive control.

Key Results

  • Significant Energy Savings: The proposed techniques achieve substantial energy reductions. Compared to a baseline with a fixed high V/F level, their "OurBest" method reduces uncore energy by approximately 80%.
  • Improved Over Prior Work: Compared to a state-of-the-art prior work (AMAT+PI [4]), their "OurBest" method provides an additional 50% power reduction.
  • Reduced Energy-Delay Product: The most impactful result is a 56% reduction in the energy-delay product compared to the state-of-the-art prior work.
  • Minimal Performance Degradation: The proposed methods achieve these energy savings with a performance loss of less than 5% across the PARSEC benchmarks.
  • Robustness to Parameters: The energy-runtime tradeoff comparison (Figure 5) shows that their "OurBest" method offers a superior Pareto-optimal curve, indicating better performance-energy efficiency across a range of parameter settings.

Limitations and Future Work

  • Control Interval Sensitivity: While the paper shows that moderate changes in control interval size don't drastically affect results, very small or very large intervals might have different impacts. The optimal interval size might also be workload-dependent.
  • Implementation Overhead: While the paper discusses implementation details, the "Throughput-driven DVFS" method has a lower overhead than latency-based methods, suggesting a potential trade-off between complexity and performance/energy gains. The direct connection requirement for the throughput method might be a scalability concern for very large CMPs.
  • Core DVFS Interaction: The paper assumes constant core frequency for simplicity. Future work could explore the interaction and co-optimization of core DVFS and uncore DVFS.
  • Workload Predictability: The supplementary material emphasizes the difficulty in accurately predicting system behavior due to its dynamic nature. While their methods are adaptive, further research into more sophisticated prediction or learning mechanisms could yield even better results.
  • Off-chip Memory Impact: While the critical latency metric attempts to exclude off-chip memory latency, its influence on overall system performance and how it interacts with uncore DVFS could be a subject for further investigation.

Summary updated

Abstract

As the core count in processor chips grows, so do the on-die, shared resources such as on-chip communication fabric and shared cache, which are of paramount importance for chip performance and power. This paper presents a method for dynamic voltage/frequency scaling of networks-on-chip and last level caches in multicore processor designs, where the shared resources form a single voltage/frequency domain. Several new techniques for monitoring and control are developed, and validated through full system simulations on the PARSEC benchmarks. These techniques reduce energy-delay product by 56% compared to a state-of-the-art prior work.
BibTeX
@misc{chen2013dynamic,
  author = {Xi Chen and Zheng Xu and Hyungjun Kim and Paul V. Gratz and Jiang Hu and Michael Kishinevsky and Ümit Y. Ogras and Raid Ayoub},
  title = {{Dynamic voltage and frequency scaling for shared resources in multicore processor designs}},
  year = {2013},
  doi = {10.1145/2463209.2488874},
}

Loading related content...

Figure