Introduction
The landscape of high-performance computing (HPC) is undergoing a fundamental transformation. As systems transition from traditional clusters toward cloud-based and exascale architectures, the methodologies used to predict and evaluate performance must evolve in tandem. This paper synthesizes the contributions from the "Performance Prediction and Evaluation" track of the Euro-Par 2012 conference, highlighting a shift from a singular focus on execution speed to a multi-dimensional view of system behavior.
The "Complexity Wall" in Modern HPC
Traditional intuition-based performance prediction is increasingly hitting a "complexity wall." The emergence of hardware heterogeneity—most notably CPU-GPU hybrids—and the massive scale of distributed systems make it nearly impossible to anticipate application behavior through simple extrapolation. Researchers now face the challenge of addressing "comprehensive concerns," where performance is no longer just about time-to-solution, but also encompasses energy efficiency, reliability, and the intricate behavior of I/O subsystems.
Methodological Innovations
The research presented in this track introduces five distinct methodologies designed to navigate this complexity:
1. Analytical Modeling for Energy Efficiency
With power consumption becoming a primary constraint in HPC design, new analytical models are required to evaluate hybrid CPU-GPU architectures. By creating metrics that specifically target energy consumption, researchers can now provide designers with the tools to balance the benefits of increased parallelism against the associated power costs.
2. Empirical Evaluation and Predictive Heuristics
In distributed environments, network latency often dictates performance. By studying the Lustre file system over Wide Area Networks (WAN), researchers have derived "rules of thumb" or heuristics. These practical guidelines allow system administrators to predict performance variations and identify potential bottlenecks caused by geographic distribution without requiring exhaustive testing.
3. Skeletal Application Generation
Isolating I/O bottlenecks is notoriously difficult in complex applications where computational logic obscures data movement. The development of "skeletal" applications—which combine high-level descriptions with low-level characterization—allows for the creation of I/O benchmarks that mimic real-world behavior. This technique enables researchers to study I/O performance in isolation, significantly reducing the overhead of full-code execution.
4. Adaptive Sampling (The ASK Kit)
The optimization space for modern hardware is vast, particularly regarding memory access patterns and stride accesses. To avoid the prohibitive cost of exhaustive measurement, the Adaptive Sampling Kit (ASK) employs multiple sampling methods to explore these spaces efficiently. This approach demonstrates that precise performance models can be constructed using a minimal number of strategic measurements.
5. Adaptive Workload Partitioning (CRAWP)
Parallel Discrete Event Simulation (PDES) is essential for designing manycore networks-on-chip. However, a unique feedback loop exists where the execution of the simulation itself is affected by the architecture being simulated. The Core/Router-Adaptive Workload Partition (CRAWP) method addresses this by making the simulation independent of the cores, improving both the speed and accuracy of simulations for benchmarks like the SPLASH2 suite.
Synthesis of Key Insights
The collective research points toward several critical shifts in the field:
- The Holistic Performance Metric: Performance is being redefined to include reliability, robustness, and energy consumption. A system's value is now measured by its ability to maintain high throughput within strict power and stability envelopes.
- Efficiency in Exploration: As hardware becomes more diverse, the "cost of exploration" must be managed. Techniques like adaptive sampling and skeletal modeling are essential for making optimization feasible in an era of near-infinite design configurations.
- Simulator Adaptability: The realization that simulators must adapt their own internal workloads to accurately reflect the hardware they model represents a significant step forward in simulation science.
Conclusion
The transition toward exascale computing requires a departure from traditional performance evaluation. The methodologies highlighted here—ranging from analytical energy models to adaptive sampling—provide a roadmap for managing the increasing diversification of parallel systems. Future work must continue to integrate qualitative modeling with quantitative measurement, ensuring that our predictive tools remain as robust as the systems they aim to optimize.
Summary updated