Abstract
This study investigates whether the resilience of serial programs can accurately predict the resilience of their parallel counterparts against soft errors. By analyzing Silent Data Corruption (SDC) rates in the NAS Parallel Benchmarks, the researchers identify scenarios where serial and parallel behaviors align and where they diverge due to parallel-specific logic and synchronization overhead.
Introduction
As high-performance computing (HPC) moves toward exascale, hardware reliability becomes a critical concern. "Soft errors"—random bit flips in a CPU's registers or memory caused by cosmic rays or electrical interference—can lead to Silent Data Corruption (SDC). In SDC scenarios, a program completes its execution but produces an incorrect result, potentially invalidating massive scientific computations.
To assess vulnerability, researchers typically use Fault Injection (FI). However, performing FI on large-scale parallel applications is computationally expensive. This paper explores a fundamental question: Can we use the results of serial program testing to predict the resilience of parallel versions?
Methodology
The research team utilized the PFSEFI (Parallel Soft Error Fault Injection) framework to conduct their experiments. The methodology focused on:
- Instruction Targeting: The study focused on floating-point addition (
fadd) and multiplication (fmul) instructions, which are central to scientific workloads. - Fault Injection Strategy: Random bit flips were injected into the results of these instructions. For parallel tests, one of four MPI processes was randomly selected for infection.
- Classification: Outcomes were categorized as Benign (correct result), SDC (incorrect result), or Crash (program failure).
- Code Mapping: Using
PYELFTOOLS, the team mapped specific memory addresses where faults occurred back to the original source code to identify vulnerability "hotspots."
Experimental Setup
The study utilized the NAS Parallel Benchmarks (NPB) suite, specifically testing three benchmarks with the "S" (small) input size:
- CG (Conjugate Gradient): Evaluates irregular memory accesses and communication.
- FT (Fast Fourier Transform): Evaluates all-to-all communication patterns.
- BT (Block Tri-diagonal solver): Evaluates computation-heavy workloads.
Analysis of Resilience
The researchers identified three distinct patterns of resilience correlation between serial and parallel executions:
The Mirror Effect (FT)
In the FT benchmark, serial and parallel versions exhibited nearly identical fault patterns. Because the core computational logic remains largely unchanged during parallelization for this algorithm, serial testing serves as an excellent proxy for parallel resilience.
The Complexity Barrier (BT)
In the BT benchmark, faulty instructions were widely scattered. The lack of a dominant computation phase or specific "hotspot" made it difficult to establish a clear correlation between the two execution modes.
Parallel-Unique Vulnerabilities (CG)
The CG benchmark revealed the most significant divergence. Parallelization introduces new variables and control flow logic (e.g., processor grid layouts) that do not exist in the serial version. Furthermore, faults in instructions dedicated to MPI synchronization can cause the parallel version to fail in ways the serial version cannot.
Key Results
The study found that SDC rates varied significantly across benchmarks and instruction types. While SDC rates for CG remained relatively low (1% to 7%), the BT benchmark showed a high vulnerability, with SDC rates reaching approximately 21%.
Mapping faults to specific addresses confirmed that while some hotspots like 0x0804A03F are universal, others like 0x0804A163 are unique to parallel execution. This confirms that parallel code possesses a distinct "vulnerability footprint."
Conclusion and Future Work
While serial resilience can predict parallel behavior in some cases (like FT), parallel-specific logic often introduces unique failure modes. This preliminary study, limited to four MPI processes and floating-point math, provides a foundation for more complex modeling. Future work aims to develop mathematical models that can accurately extrapolate parallel resilience from serial data, potentially saving significant compute time in the development of fault-tolerant exascale applications.
Summary updated