Energy-Efficient QoS-Aware Scheduling for S-NUCA Many-Cores

Sudam Maduranga Wasala, Jurre Wolff, Yixian Shen, Anuj Pathania, Clemens Grelck, Andy D. Pimentel

TL;DR

This paper introduces a novel Quality of Service (QoS) management policy for S-NUCA many-core processors that uses application-specific "heartbeats" to maintain performance targets. The policy dynamically adjusts thread placement and core frequencies to meet QoS, then optimizes for energy efficiency, achieving significant energy savings without compromising performance.

What Problem Does This Paper Address?

Modern many-core processors, especially those with Non-Uniform Cache Access (NUCA) architectures, present a significant challenge for performance and energy efficiency. In S-NUCA systems, the last-level cache (LLC) is physically distributed, meaning accessing data from different cache banks incurs varying latencies depending on a core's proximity to the center of the chip. This inherent heterogeneity complicates task scheduling, as simply aiming for high Instructions Per Second (IPS) doesn't guarantee consistent application performance, especially for latency-sensitive applications like games where frame rates are critical. Traditional metrics can be misleading, and there's a growing need for application-specific performance indicators.

Methods

The authors developed a novel QoS management policy that leverages the S-NUCA architecture's heterogeneity. The core of their approach is the "Application Heartbeats" framework, which allows applications to report their real-time performance. This framework is integrated into the HotSniper simulator, enabling precise, application-specific performance monitoring.

The QoS management policy has three main objectives:

  1. Achieve QoS: Maintain the application's performance within a predefined target "heart rate" (HR).
  2. Minimize Jitter: Reduce fluctuations in HR outside the target range.
  3. Optimize Energy: Minimize energy consumption once QoS is met.

The policy uses two primary "knobs" to control performance:

  • Thread Migration: Moving threads to cores with lower average cache access latencies (closer to the chip's center).
  • DVFS (Dynamic Voltage and Frequency Scaling): Adjusting core clock frequencies to influence processing speed and power consumption.

The policy operates reactively, using a hierarchical decision-making process based on the current HR relative to a target range. It classifies the system's state into five categories (A-E) and then selects primary and secondary actions (frequency adjustment or thread migration) to bring the HR back into the target range. Once the HR is stable within the target, the policy shifts to energy optimization, carefully managing frequency to balance power consumption and performance.

Datasets

The authors evaluated their policy using the PARSEC benchmark suite, specifically the blackscholes, canneal, and dedup applications with simsmall inputs. They simulated a 64-core S-NUCA architecture with specific cache and network-on-chip (NoC) configurations. No new benchmarks were introduced; instead, existing ones were adapted to use the heartbeat framework.

Key Insights and Ideas

  • Application Heartbeats are Crucial for QoS: Traditional metrics like IPS are insufficient for guaranteeing predictable application performance. Application-specific metrics like heartbeats provide a more accurate and actionable way to monitor and manage QoS.
  • Exploiting S-NUCA Heterogeneity is Key: The spatial distribution of caches in S-NUCA systems isn't just a challenge; it's an opportunity. The policy effectively uses thread migration to leverage cores with lower access latencies.
  • Reactive Control is More Robust: Instead of complex predictive models, a reactive approach that responds to real-time HR measurements proves more adaptable to the dynamic nature of application workloads and thread counts.
  • Hierarchical Control Balances Speed and Precision: The policy intelligently combines macro (large) and micro (small) frequency adjustments with thread migration, prioritizing faster HR changes when needed and finer-grained control for stability and energy optimization.
  • Energy Optimization Follows QoS: The policy prioritizes meeting performance targets before aggressively pursuing energy savings, ensuring that QoS is never sacrificed for power efficiency.

Key Results

The proposed QoS management policy demonstrated significant improvements:

  • Effective QoS Maintenance: The policy consistently maintained the application's Heart Rate (HR) within the predefined target range across various benchmarks and thread counts.
  • Reduced Oscillations: Compared to state-of-the-art methods like HPM and PCMig, the proposed policy showed a significant reduction in HR oscillations.
  • Energy Savings: The approach achieved 18.7% energy savings compared to state-of-the-art scheduling methods. While PCMig sometimes showed better energy performance, it often failed to meet QoS targets.

Limitations and Future Work

  • Simulation-Based Evaluation: The current work relies on simulations using HotSniper. The authors suggest future work to implement and test the policy on a real hardware system.
  • Thermal Constraints Not Considered: The current policy does not account for thermal management. Future work will involve integrating thermal strategies into the QoS management policy.
  • Prohibitively Long Execution Times: Some benchmarks had prohibitively long execution times in simulations, suggesting that real-world testing might reveal different challenges or require further optimization for such workloads.
  • Unified Frequency for Application Threads: For simplicity, the policy enforces a unified frequency for all cores running threads of the same application. This might not be optimal in all scenarios and could be a point for future exploration.

Summary updated

Abstract

Optimizing performance and energy efficiency in many-core processors, especially within Non-Uniform Cache Access (NUCA) architectures, remains a critical challenge. The performance heterogeneity inherent in S-NUCA systems complicates task scheduling due to varying cache access latencies across cores. This paper introduces a novel QoS management policy to maintain application execution within predefined Quality of Service (QoS) targets, measured using the Application Heartbeats framework. QoS metrics like Heartbeats ensure predictable application performance in dynamic computing environments. The proposed policy dynamically controls QoS by orchestrating task migrations within the S-NUCA many-core system and adjusting the clock frequency of cores. After satisfying the QoS objectives, the policy optimizes energy efficiency, reducing overall system energy consumption without compromising performance constraints. Our work leverages the state-of-the-art multi-/many-core simulator HotSniper. We have extended it with two key components: an integrated heartbeat framework for precise, application-specific performance monitoring, and our QoS management policy that maintains application QoS requirements while minimizing the system's energy consumption. Experimental evaluations demonstrate that our approach effectively maintains desired QoS levels and achieves 18.7% energy savings compared to state-of-the-art scheduling methods.
BibTeX
@inproceedings{wasala2025energy,
  author = {Sudam Maduranga Wasala and Jurre Wolff and Yixian Shen and Anuj Pathania and Clemens Grelck and Andy D. Pimentel},
  title = {{Energy-Efficient QoS-Aware Scheduling for S-NUCA Many-Cores}},
  booktitle = {ISQED},
  year = {2025},
  url = {https://openreview.net/forum?id=V2jFutjVrl},
}
Figure