SCOPE: Safe Exploration for Dynamic Computer Systems Optimization

Hyunji Kim, Ahsan Pervaiz, Henry Hoffmann, Michael Carbin, Yi DingarXiv 2022 · DOI · arXiv

Abstract

Optimizing modern computer systems requires balancing peak performance with strict safety constraints, such as power caps. Traditional machine learning approaches often rely on offline training, which fails to account for the dynamic nature of workloads where safe configurations do not generalize across different applications or inputs. We present SCOPE (Safe Configuration Optimization for Performance and Efficiency), a resource manager that safely explores the hardware configuration space at runtime. By employing Gaussian Process Regression and a novel locality-preserving operator, SCOPE significantly improves system latency while maintaining near-zero safety violations.

Introduction

The management of modern data centers involves a persistent conflict between maximizing performance (minimizing latency) and adhering to safety constraints (staying under power thresholds). Exceeding power limits can lead to hardware degradation or catastrophic failures like circuit breaker trips. While machine learning has been proposed for system optimization, most existing solutions are trained offline. This research identifies a critical limitation: safe configurations are highly workload-dependent. A hardware setting that is safe for one task may cause a dangerous power spike in another.

To address this, the authors propose SCOPE, a framework designed to explore new hardware configurations during execution. SCOPE treats optimization as a search through an "execution space"—a complex domain encompassing hardware resources, input features, and application characteristics.

The SCOPE Framework

SCOPE operates as an iterative control loop consisting of three distinct phases: Monitoring, Safe Space Construction, and Objective Optimization.

The SCOPE iterative optimization loop, featuring monitoring, safe space construction, and objective optimization phases.
  1. Monitoring: The system continuously measures power consumption and performance metrics. If a safety violation occurs, the system immediately triggers a re-evaluation.
  2. Safe Space Construction: This phase is the core of SCOPE's safety mechanism. It uses Gaussian Process Regression (GPR) to model the safety boundary of the system. To prevent catastrophic jumps into unsafe regions, SCOPE introduces a Locality Preserving Operator. This operator restricts exploration to configurations "near" the current safe state, leveraging the observation that small hardware adjustments (e.g., incremental changes in CPU frequency or core count) result in predictable, incremental changes in power.
  3. Objective Optimization: Once a "safe set" of configurations is predicted, SCOPE selects the one expected to provide the best performance (e.g., the lowest latency) and reconfigures the hardware accordingly.

Experimental Evaluation

The researchers evaluated SCOPE using 12 Apache Spark applications from the HiBench suite, including machine learning, graph analytics, and web search workloads. Testing was performed on the Chameleon cloud platform using dual-socket Intel Xeon Gold nodes, with an execution space of 1,920 possible hardware configurations.

A key finding of the study is the volatility of safe sets. As shown in the figure below, a configuration optimized for one workload (e.g., ALS) can lead to significant power threshold violations when applied to another (e.g., N-weight). This highlights why static or offline-only models are insufficient for modern dynamic systems.

Power consumption over time for different workloads, demonstrating that safe configurations for one application (als) lead to violations when applied to another (nweight).

Key Results and Insights

SCOPE demonstrates substantial improvements over baseline methods like Intel’s RAPL and standard Bayesian Optimization (BO):

  • Performance: SCOPE achieved up to 9.5× better latency than RAPL, which prioritizes power capping over performance optimization.
  • Safety: By using the locality operator, SCOPE reduced power constraint violations by up to 45.88× compared to standard BO. It maintained near-zero violations throughout the testing period.
  • Exploration Efficiency: The locality-preserving approach allowed for 1.96× higher coverage of the safe set and 1.35× lower violation magnitude than versions of the algorithm without this constraint.
  • Low Overhead: The computational cost of SCOPE is approximately 0.05s per sample, making it highly suitable for real-time resource management in large-scale data tasks.

Conclusion and Future Work

SCOPE proves that "locality" is a powerful tool for safe runtime exploration in discrete, non-linear computer systems. By adopting a "crawl-before-you-run" strategy, systems can safely navigate complex configuration spaces.

Future research directions include providing formal deterministic safety guarantees, as SCOPE currently relies on probabilistic models. Additionally, the authors suggest making hyperparameters like the exploration radius adaptive, allowing the system to expand its search more aggressively when model confidence is high. Porting the implementation to lower-level languages could also further minimize management overhead.

Summary updated

Abstract

Modern computer systems need to execute under strict safety constraints (e.g., a power limit), but doing so often conflicts with their ability to deliver high performance (i.e. minimal latency). Prior work uses machine learning to automatically tune hardware resources such that the system execution meets safety constraints optimally. Such solutions monitor past system executions to learn the system's behavior under different hardware resource allocations before dynamically tuning resources to optimize the application execution. However, system behavior can change significantly between different applications and even different inputs of the same applications. Hence, the models learned using data collected a priori are often suboptimal and violate safety constraints when used with new applications and inputs. To address this limitation, we introduce the concept of an execution space, which is the cross product of hardware resources, input features, and applications. To dynamically and safely allocate hardware resources from the execution space, we present SCOPE, a resource manager that leverages a novel safe exploration framework. We evaluate SCOPE's ability to deliver improved latency while minimizing power constraint violations by dynamically configuring hardware while running a variety of Apache Spark applications. Compared to prior approaches that minimize power constraint violations, SCOPE consumes comparable power while improving latency by up to 9.5X. Compared to prior approaches that minimize latency, SCOPE achieves similar latency but reduces power constraint violation rates by up to 45.88X, achieving almost zero safety constraint violations across all applications.
BibTeX
@misc{kim2022scope,
  author = {Hyunji Kim and Ahsan Pervaiz and Henry Hoffmann and Michael Carbin and Yi Ding},
  title = {{SCOPE: Safe Exploration for Dynamic Computer Systems Optimization}},
  year = {2022},
  eprint = {2204.10451},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.10451},
}
Figure