Characterization and Comparison of Application Resilience for Serial and Parallel Executions

Kai Wu, Qiang Guan, Nathan DeBardeleben, Dong LiarXiv 2018 · arXiv

Abstract

This study investigates whether the resilience of serial programs can accurately predict the resilience of their parallel counterparts against soft errors. By analyzing Silent Data Corruption (SDC) rates in the NAS Parallel Benchmarks, the researchers identify scenarios where serial and parallel behaviors align and where they diverge due to parallel-specific logic and synchronization overhead.

Introduction

As high-performance computing (HPC) moves toward exascale, hardware reliability becomes a critical concern. "Soft errors"—random bit flips in a CPU's registers or memory caused by cosmic rays or electrical interference—can lead to Silent Data Corruption (SDC). In SDC scenarios, a program completes its execution but produces an incorrect result, potentially invalidating massive scientific computations.

To assess vulnerability, researchers typically use Fault Injection (FI). However, performing FI on large-scale parallel applications is computationally expensive. This paper explores a fundamental question: Can we use the results of serial program testing to predict the resilience of parallel versions?

Methodology

The research team utilized the PFSEFI (Parallel Soft Error Fault Injection) framework to conduct their experiments. The methodology focused on:

  • Instruction Targeting: The study focused on floating-point addition (fadd) and multiplication (fmul) instructions, which are central to scientific workloads.
  • Fault Injection Strategy: Random bit flips were injected into the results of these instructions. For parallel tests, one of four MPI processes was randomly selected for infection.
  • Classification: Outcomes were categorized as Benign (correct result), SDC (incorrect result), or Crash (program failure).
  • Code Mapping: Using PYELFTOOLS, the team mapped specific memory addresses where faults occurred back to the original source code to identify vulnerability "hotspots."

Experimental Setup

The study utilized the NAS Parallel Benchmarks (NPB) suite, specifically testing three benchmarks with the "S" (small) input size:

  1. CG (Conjugate Gradient): Evaluates irregular memory accesses and communication.
  2. FT (Fast Fourier Transform): Evaluates all-to-all communication patterns.
  3. BT (Block Tri-diagonal solver): Evaluates computation-heavy workloads.

Analysis of Resilience

The researchers identified three distinct patterns of resilience correlation between serial and parallel executions:

The Mirror Effect (FT)

In the FT benchmark, serial and parallel versions exhibited nearly identical fault patterns. Because the core computational logic remains largely unchanged during parallelization for this algorithm, serial testing serves as an excellent proxy for parallel resilience.

The Complexity Barrier (BT)

In the BT benchmark, faulty instructions were widely scattered. The lack of a dominant computation phase or specific "hotspot" made it difficult to establish a clear correlation between the two execution modes.

Parallel-Unique Vulnerabilities (CG)

The CG benchmark revealed the most significant divergence. Parallelization introduces new variables and control flow logic (e.g., processor grid layouts) that do not exist in the serial version. Furthermore, faults in instructions dedicated to MPI synchronization can cause the parallel version to fail in ways the serial version cannot.

Distribution of SDC-causing instruction addresses in a parallel execution environment showing unique vulnerability footprints.

Key Results

The study found that SDC rates varied significantly across benchmarks and instruction types. While SDC rates for CG remained relatively low (1% to 7%), the BT benchmark showed a high vulnerability, with SDC rates reaching approximately 21%.

Comparison of SDC rates for fadd and fmul instructions across serial and parallel benchmarks.

Mapping faults to specific addresses confirmed that while some hotspots like 0x0804A03F are universal, others like 0x0804A163 are unique to parallel execution. This confirms that parallel code possesses a distinct "vulnerability footprint."

Conclusion and Future Work

While serial resilience can predict parallel behavior in some cases (like FT), parallel-specific logic often introduces unique failure modes. This preliminary study, limited to four MPI processes and floating-point math, provides a foundation for more complex modeling. Future work aims to develop mathematical models that can accurately extrapolate parallel resilience from serial data, potentially saving significant compute time in the development of fault-tolerant exascale applications.

Summary updated

BibTeX
@misc{wu2018characterization,
  author = {Kai Wu and Qiang Guan and Nathan DeBardeleben and Dong Li},
  title = {{Characterization and Comparison of Application Resilience for Serial and Parallel Executions}},
  year = {2018},
  eprint = {1808.01093},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1808.01093},
}
Figure