TL;DR
This paper investigates how compiler optimizations, a standard practice for boosting GPU performance, can surprisingly impact their reliability. By combining hardware fault injection, performance profiling, and neutron beam experiments, the authors demonstrate that while optimizations can increase error rates, the performance gains often lead to a higher overall amount of useful work completed before failure.
What Problem Does This Paper Address?
Modern GPUs are powerful parallel processors used in critical applications like High-Performance Computing (HPC) and autonomous driving. To maximize their performance, developers rely heavily on compiler optimizations. However, these optimizations alter how code is mapped to the GPU's hardware, potentially affecting its susceptibility to errors, especially transient faults common in GPUs. Existing research often focuses on either software-level fault injection or hardware-level beam experiments, but a comprehensive understanding of how compiler optimizations bridge these two aspects and influence GPU reliability has been lacking. This paper aims to quantify and understand this impact, addressing the critical need for reliable GPU code in safety-conscious domains.
Methods
The authors employ a multi-layered approach to assess GPU reliability under compiler optimizations:
- Software Fault Injection (NVBitFI): This technique simulates transient errors at the instruction set architecture (SASS) level. It measures the Program Vulnerability Factor (PVF), which indicates how likely a fault in a specific instruction is to propagate to an output error (Silent Data Corruption - SDC, or Detected Unrecoverable Error - DUE).
- Instruction Profiling: This captures the dynamic behavior of the code, including instruction distribution and resource utilization. This helps understand how optimizations change the code's mapping to hardware.
- SDC Probability (P(SDC)): This metric combines PVF with instruction distribution to estimate the probability of an SDC occurring, considering which instructions are more likely to be executed.
- SDC FIT Estimation: This advanced metric integrates hardware fault probability (measured via beam experiments) with software analysis (PVF and instruction distribution) to provide a more realistic estimate of the error rate (Failure In Time - FIT).
- Neutron Beam Experiments: These experiments expose real GPUs to neutron radiation to directly measure the hardware fault probability and validate the predicted error rates. This is considered the most accurate method for assessing real-world reliability.
Datasets
The paper evaluates ten representative benchmarks from various domains, including:
- Breadth-First Search (BFS)
- Lava MD (LVA)
- Hotspot (HST)
- Gaussian elimination (GSS)
- LU Decomposition (LUD)
- CFD Solver (CFD)
- General Matrix Multiplication (GEMM)
- Merge Sort (MST)
These benchmarks were compiled using two versions of the NVIDIA CUDA Compiler (NVCC 10.2 and 11.3) and various optimization flags. No new benchmarks were introduced; existing ones were used to ensure broad applicability.
Key Insights and Ideas
- Optimization Trade-off: Compiler optimizations for performance often come with a trade-off: they can increase the GPU's error rate. However, the performance gains are frequently substantial enough to outweigh this increase, leading to more useful work completed before a failure.
- Hardware Sensitivity is Key: The impact of compiler optimizations on reliability is heavily influenced by the underlying hardware's sensitivity to faults. Simply analyzing software-level fault propagation (like SDC probability) is insufficient; incorporating hardware fault rates is crucial for accurate estimations.
- Compiler Versions Matter: Different NVCC versions can generate code with varying reliability characteristics, even for the same optimization flags. This highlights the importance of considering compiler updates in reliability assessments.
- Instruction Type Influence: Codes that heavily rely on specific instruction types (e.g., floating-point arithmetic like FMA/FMUL) tend to have higher error propagation probabilities, as small deviations can cascade through repetitive operations.
- Unoptimized Code Masks Faults (but is slower): Unoptimized code (O0) can sometimes mask faults due to redundant instructions or dead code. However, this comes at the cost of significantly lower performance, meaning less work is done before a failure.
Key Results
- Order of Magnitude Impact: Compiler optimizations can impact the GPU's error rate by up to an order of magnitude.
- Performance Gain Outweighs Error Increase: For most applications, the performance gain from optimizations is significantly higher than the increase in error rate, leading to a higher Mean Work Between Failures (MWBF). For example, optimized versions often produce double the amount of correct data before failure compared to unoptimized code.
- NVCC 10.2 vs. 11.3: The SDC FIT estimation shows NVCC 10.2 can be up to 8x higher than NVCC 11.3 for certain configurations, a discrepancy not seen in SDC probability alone, underscoring the role of hardware fault rates.
- GEMM Example: Neutron beam experiments on GEMM showed that while optimizations can increase FIT rates, the MWBF for optimized versions (O1, O3) was consistently higher than for less optimized ones (MinRF, O0), indicating more reliable overall output.
- ECC's Role: Disabling ECC significantly increases the SDC FIT rate, as unprotected register files and caches become primary error sources.
Limitations and Future Work
- Microinstruction Granularity: Measuring the error rate for every single microinstruction is unfeasible due to beam time limitations. The study groups microinstructions, which limits the finest-grained analysis.
- Proprietary Compiler: The proprietary nature of NVCC restricts a complete analysis of the machine binary.
- Focus on Functional Units: The SDC probability and SDC rate estimation in this study primarily consider faults in functional units and output registers, excluding caches and shared memories.
- Future Work: The authors suggest that future work could explore the impact of compiler optimizations on other GPU components like caches and shared memories. They also emphasize the need to consider application priorities (performance vs. reliability) when selecting optimization strategies. The study also points to the need for more detailed analysis of specific instruction types and their interaction with hardware sensitivities.
Summary updated